Koozali.org: home of the SME Server

Installing Contribs in sme without internet access

Offline mealogav

  • *
  • 27
  • +0/-0
Re: Installing Contribs in sme without internet access
« Reply #15 on: June 30, 2013, 07:54:01 PM »
Ok - I take your point.  I made a new download of sme8 yesterday which I used to install on this server from scratch.  I assumed that this would be an "up to date" starting point - but obviously not.  I notice the current sme 8 download has a date of 25 May 2012 - so ages ago.

My problem is that the "production" server that I need to install mcrypt and phpMyAdmin to was installed in Feb 2013 from a fresh sme8 download, so it will be equally in need of updating.  BUT it is in a location that does not have any internet access and is only used on that local network.

I've now tested the installation on my home server with internet access and I'm still puzzled why even though the yum command includes the "localinstall " option, it still goes off to the mirror sites to look for updates??

As far as I can tell the installation has worked - the mcrypt library certainly shows up in a phpinfo listing.  Although I think phpMyAdmin has installed I can't work out how to access it??  I expected:
http://hostname/phpMyAdmin   (or https://hostname/phpMyAdmin ) to work, but it doesn't.  Any clues?

You have been a great help - I now know that to get these modules installed I'll somehow need to get temporary internet access!  (I can't even physically move the server as it's running on a VM server with 11 other servers!!)

Thanks anyway.  Gavin

Offline TerryF

  • grumpy old man
  • *
  • 1,821
  • +6/-0
Re: Installing Contribs in sme without internet access
« Reply #16 on: July 01, 2013, 02:09:48 AM »
A new install of SME8 needs a "yum update" to bring it up to date, currently that sits at approx 250mb plus and approx 160 plus packages, even with those updates ongoing upstream updates are a regular event as well as just SME specific changes.

See HERE for bug requesting a new iso and HERE for list discussion on same.

Even with a brand new iso incorporating all updates as at today, there will still be updates coming down the pipe from upstream as well as SME specific within a very short time.

Added later: Just noticed THIS post re new iso
« Last Edit: July 01, 2013, 02:34:59 AM by TerryF »
--
qui scribit bis legit

Offline mealogav

  • *
  • 27
  • +0/-0
Re: Installing Contribs in sme without internet access
« Reply #17 on: July 01, 2013, 09:40:39 AM »
Wow! - Just started an update - 245 Mb to download.  They are right in the bug request "this doesn't create a good impression for new installers"!
Maybe this is why phpMyAdmin isn't working on my system?  Maybe it needs an update - we'll see ...

I think this just focuses my attention on just how difficult it is to install a couple of Contribs on a server that has no internet access!  Thanks again for your help.  Gavin

Offline janet

  • ****
  • 4,812
  • +0/-0
Re: Installing Contribs in sme without internet access
« Reply #18 on: July 01, 2013, 02:50:28 PM »
mealogav

Neither Life or SME server work magically.

You need to use a Windows etc workstation that is connected to the Internet, download all the required contrib packages & dependency packages, copy them to USB, then move the files to a folder or ibay on your SME server
Mount the USB stick like you would mount any other drive (refer Howtos) & copy the files to the folder or ibay

Then run the command
yum localinstall packagename

It should find all the dependencies in the same folder.
If anything is missing it will let you know.
Repeat the above process if necessary, making sure to get any dependencies you originally missed.

....And yes, part of this procedure to install recently upgraded contribs, is to keep your sme server up to date also, using similar techniques as above.


It may be easier to setup a test SME server at home where you have Internet access, & use that to download & install contribs, in the process you will learn which rpms & dependency rpms to manually download using the wget command. Then transfer all the files from the test server to VM server using USB etc.

There is no other magic answer.
Please search before asking, an answer may already exist.
The Search & other links to useful information are at top of Forum.

Offline Stefano

  • *
  • 10,836
  • +2/-0
Re: Installing Contribs in sme without internet access
« Reply #19 on: July 01, 2013, 04:46:36 PM »
Wow! - Just started an update - 245 Mb to download.  They are right in the bug request "this doesn't create a good impression for new installers"!
Maybe this is why phpMyAdmin isn't working on my system?  Maybe it needs an update - we'll see ...

I think this just focuses my attention on just how difficult it is to install a couple of Contribs on a server that has no internet access!  Thanks again for your help.  Gavin

please re-read carefully this post:
http://forums.contribs.org/index.php/topic,49707.msg248693.html#msg248693

you should really point to the "local yum repository" link

you already had all the answers..

Offline mealogav

  • *
  • 27
  • +0/-0
Re: Installing Contribs in sme without internet access
« Reply #20 on: July 01, 2013, 07:18:39 PM »
Hi Janet - thanks for persevering with me ...
Quote
It may be easier to setup a test SME server at home where you have Internet access, & use that to download & install contribs
That's exactly what I was trying to do this weekend and follow your instructions from 18th March 2013 http://forums.contribs.org/index.php/topic,49707.msg248691.html#msg248691

However - what I've discovered over the weekend is that a new sme8 iso image download requires over 300 packages and 245 Mb to bring it up to date.  Some of those are necessary to satisfy dependancies during the phpMyAdmin and mcrypt contribs install.

I downloaded the packages that I thought were needed, as advised by stephdl on 17 March, copied them to an iBay and installed from there.
smeserver-phpmyadmin-3.5.2.2-6.el5.sme.noarch.rpm
phpMyAdmin3-3.5.8.1-1.el5.noarch.rpm
php53-php-gettext-1.0.11-3.el5.noarch.rpm
php53-mcrypt-5.3.3-1.el5.i386.rpm
libmcrypt-2.5.8-4.el5.centos.i386.rpm

BUT - as the production server (and initially my home server), had never been updated from the 25 May 2012 version that is still the iso available for download, what this flagged up is that there are loads of other updates needed.  Even though I did a localinstall, I Iassume the packages available locally were used, and yum tried to find other not available locally from the mirror sites.

I know there is no "magic" - and I am learning alot through this exercise (just not enough!).

I gave up on the local install on my home server and plugged the internet connection back in and let it update and install the contribs.
mcrypt seems to have installed ok (it shows in a phpinfo listing), but although it apparently installed ok phpMyAdmin isn't working yet.  I can see evidence of it in apache config files - but I have to look into this a bit more.

So thanks for your help.  I think some of the more sophisticated answers (that I don't yet understand) are the real answer for my situation - like creating a local repository ...  see next post in answer to Stephano.

Gavin

Offline mealogav

  • *
  • 27
  • +0/-0
Re: Installing Contribs in sme without internet access
« Reply #21 on: July 01, 2013, 07:43:56 PM »
Hi Stefano
Quote
please re-read carefully this post:
http://forums.contribs.org/index.php/topic,49707.msg248693.html#msg248693

Reading carefully is not the problem - trust me I've read all the posts Very carefully.  Understanding them is the problem.
For example :
Quote
Setup a virtual copy of your SME server
What does this mean?  Does it just mean a different copy of sme server on a computer with internet access?  Or is there something significant in the "virtual" description?  If it means set up a virtual server (eg VMWare) and create a new sme server virtual machine, well sorry but your average sme home user doesn't have that kind of facility available!  However I think this could be just a server with internet access - right?
Quote
Setup a yum local repository
If I understand this "How To" correctly, this is going to create a copy of the sme repository?  ie it's going to download all packages to a local directory on the local server Right?
Then if I copy this to my server with no internet access and issue cd to the root directory of this repository (the ibay directory), when I issue yum localinstall commands it should find all the local packages - right?

I assume that I could expand this principle and if I omitt the appropriate "exclude" options from the rsync command used to create the repository then contribs will be downloaded and can be installed from this local repos - right?

Thanks - I do really appreciate the help.  Gavin

Offline mealogav

  • *
  • 27
  • +0/-0
Re: Installing Contribs in sme without internet access
« Reply #22 on: July 10, 2013, 11:38:05 AM »
I now have a "test" sme server that I can disconnect from the internet to develop this technique further.
I've follow the suggestion mmccarn made in forum post :
http://forums.contribs.org/index.php/topic,49707.msg248693.html#msg248693
to set up a local yum repository and install from there.

I've followed the instructions at  http://wiki.contribs.org/Yum_local_repository
and I have the local repos in a repos ibay on my normal hoem server sme machine.  That all seems to have worked well and I can browse to it at  http://192.168.11.2/repos/  and follow the links to
http://192.168.11.2/repos/releases/8/smeupdates/i386/repoview/  and see a html page of packages.

(Because after it has updated I want to install phpMyAdmin on my server that has no internet access,
I removed some of  the exclusions from the rsync command so I actually have
repos/releases/8/smeupdates
repos/releases/8/smeaddons
repos/releases/8/smeextras
repos/releases/8/contribs
about 1.5GB)
So far so good ...

So on my separate sme server (without internet access), I do a fresh sme8 install from the iso, then follow the instructions for the "client"
[root@dell files]# /sbin/e-smith/db yum_repositories setprop smeupdates status disabled
ok
I modified the repository set command to suit sme8 (the example is for 7 and seems to have a spurious smeserver directory that doesn't exist on the sme8 mirror that I downloaded from - http://distro.ibiblio.org/ )
from  BaseURL 'http://<yourServersIP or FQDN>/repos/smeserver/releases/7/smeupdates/$basearch' \
to BaseURL 'http://192.168.11.2/repos/releases/8/smeupdates/$basearch' \
[root@dell files]# /sbin/e-smith/db yum_repositories set smeupdates-local repository \
>  Name 'SME Server - updates (local)' \
>  BaseURL 'http://192.168.11.2/repos/releases/8/smeupdates/$basearch' \
>  EnableGroups yes \
>  GPGCheck yes \
>  Visible yes \
>  status enabled
[root@dell files]# signal-event yum-modify
ok so far ... BUT
when I try the
yum clean all   command I get the following message that repeats every 2 seconds ...:
Loaded plugins: fastestmirror, protect-packages, smeserver
Existing lock /var/run/yum.pid: another copy is running as pid 3700.
Another app is currently holding the yum lock; waiting for it to exit...
  The other application is: yum
    Memory :  13 M RSS ( 22 MB VSZ)
    Started: Wed Jul 10 10:08:35 2013 - 01:06 ago
    State  : Sleeping, pid: 3700
Another app is currently holding the yum lock; waiting for it to exit...
  The other application is: yum
    Memory :  13 M RSS ( 22 MB VSZ)
    Started: Wed Jul 10 10:08:35 2013 - 01:08 ago
    State  : Sleeping, pid: 3700
I assume this is yum re-configuring??  If you wait (about 20 mins) and repeat the command it seems to work ...

[root@dell files]# yum clean all
Loaded plugins: fastestmirror, protect-packages, smeserver
Cleaning up Everything
Cleaning up list of fastest mirrors
[root@dell files]#

I assume that in server-manager when I go to the "Software Installer" page the updates should now be located from the repos on the
192.168.11.2  server?  Is that correct? 

What I get is the normal message saying that "Updates are available" but when I click to install them in the scroll list that would normally show packages to be updated is the message:
Quote
Could not - retrieve
[Errno 4] - IO Error:

I've tried to check the setting of the $basearch variable by using "echo".  It appears to be empty??  Is this a concern?  I'm only guessing, but I expected this to contain the cpu architecture eg "i386"
Quote
[root@dell files]# echo $basearch

[root@dell files]#

I'm sorry to be such a pain about this but I feel I'm very close now and just need a final nudge in the right direction.
Thanks for any further help.  Gavin

Offline mealogav

  • *
  • 27
  • +0/-0
Re: Installing Contribs in sme without internet access
« Reply #23 on: July 10, 2013, 12:29:14 PM »
Even though there was just an error showing instead of packages, I tried clicking "Install" to see what messages I got ...

Quote
Loaded plugins: fastestmirror, protect-packages, smeserver
Loading mirror speeds from cached hostfile
Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=5&arch=i386&repo=os error was
[Errno 4] IOError:
Error: Cannot find a valid baseurl for repo: base
yum update Could failed: 256
The system needs to be reconfigured after adding or removing software, or applying updates.

I think this suggests that it's not trying to use the local repos on the other (local to my house at least) machine?  Any clues why?

As an alternative, now that I have a local copy of all the rpm files etc in the repository, if I simply copied them to, for example, an ibay on the sme server that doesn't have internet access, what command would I issue to get the server to update all of sme from those files?
I know that for contribs I would copy the necessary rpms to a directory, cd to that ddirectory and issue something like :
"yum localinstall smeserver-phpmyadmin3"
"signal-event post-upgrade;signal-event reboot"

But what would the yum command be to get it to update all of sme server?

Thanks.

Offline TerryF

  • grumpy old man
  • *
  • 1,821
  • +6/-0
Re: Installing Contribs in sme without internet access
« Reply #24 on: July 10, 2013, 01:53:56 PM »
Issue this from a command line

 db yum_repositories show

It will list all your repos and where they are pointing to
--
qui scribit bis legit

Offline mealogav

  • *
  • 27
  • +0/-0
Re: Installing Contribs in sme without internet access
« Reply #25 on: July 10, 2013, 02:49:26 PM »
Thanks Terry - but my question is given that I now have a copy of the repository(s) on another machine accessible on my local network at
http://192.168.11.2/repos/releases/8/smeupdates/
http://192.168.11.2/repos/releases/8/smeupdates/i386
http://192.168.11.2/repos/releases/8/smeupdates/x86_64   etc etc

   what do the yum db repository entries need to be set to in order that sme will do a software update
from the server-manager "Software Update" page?

All the setting that seem obvious to me haven't worked yet - so there's obviously something (many things!) I don't yet understand.

Thanks.

Offline mmccarn

  • *
  • 2,626
  • +10/-0
Re: Installing Contribs in sme without internet access
« Reply #26 on: July 10, 2013, 04:35:36 PM »
You are right in your assumption about 'signal-event yum-modify' - it kicks off yum in the background in order to obtain the latest package information for display in server-manager.

db_yumrepositories show
* would tell us how your repos are currently configured. 
* would tell us if there were any unexpected errors during creation of the new repos (copy & paste may have resulted in $basearch being replaced by the null string you are seeing in bash, for example)
* would allow us to comment on which repos are enabled and which are disabled
...etc.

Note that if you have internet access from ANY computer that the SME can get to, you could configure yum to use a proxy and just run updates from the web. Of course, it might take quite a bit of jiggery-pokery to get a proxy server running on your network somewhere...

Another option - get a mobile hotspot and turn that on when you want to do updates and install software ;-)





Offline mealogav

  • *
  • 27
  • +0/-0
Re: Installing Contribs in sme without internet access
« Reply #27 on: July 10, 2013, 07:03:37 PM »
Thanks chaps - one of the last things I did before giving up was look at the
  db yum_repositories show
output - but then I had to go to work.  I'll post it here later.

Thanks Gavin

Offline mealogav

  • *
  • 27
  • +0/-0
Re: Installing Contribs in sme without internet access
« Reply #28 on: July 11, 2013, 12:53:47 AM »
Ok - here's the db yum_repositories show   output.

A couple of comments.
The how to at   http://wiki.contribs.org/Yum_local_repository sets a new repos called "smeupdates-local".  I'm just wondering now how can this possibly change the behaviour of the "normal" updating operation? (ie what you get when using server-manager Software Installer page).  Wouldn't you have to change the existing "smeupdates" repos (and maybe some others?), I mean what's the mechanism that will mean that yum includes a new repository in it's search path - without including it in /etc/yum.repos.d/  ?

I've messed about with the original database entry for the smeupdates repos (smeupdates=repository) in an attempt to get it working.  So this might not be exactly as the original db entry.

[root@dell ~]# /sbin/e-smith/db yum_repositories show
addons=repository
    EnableGroups=no
    GPGCheck=yes
    MirrorList=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=addons
    Name=CentOS - addons
    Visible=no
    status=disabled
base=repository
    EnableGroups=no
    Exclude=initscripts,libgsf
    GPGCheck=yes
    MirrorList=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os
    Name=CentOS - os
    Visible=yes
    status=enabled
centosplus=repository
    EnableGroups=no
    GPGCheck=yes
    MirrorList=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplus
    Name=CentOS - centosplus
    Visible=no
    status=disabled
contrib=repository
    EnableGroups=no
    GPGCheck=yes
    MirrorList=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=contrib
    Name=CentOS - contrib
    Visible=no
    status=disabled
extras=repository
    EnableGroups=no
    GPGCheck=yes
    MirrorList=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras
    Name=CentOS - extras
    Visible=no
    status=disabled
smeaddons=repository
    EnableGroups=yes
    GPGCheck=yes
    MirrorList=http://distro.ibiblio.org/pub/linux/distributions/smeserver/mirrorlist/smeaddons-8
    Name=SME Server - addons
    Visible=yes
    status=enabled
smecontribs=repository
    EnableGroups=yes
    GPGCheck=yes
    MirrorList=http://distro.ibiblio.org/pub/linux/distributions/smeserver/mirrorlist/smecontribs-8
    Name=SME Server - contribs
    Visible=no
    status=disabled
smedev=repository
    EnableGroups=yes
    GPGCheck=yes
    MirrorList=http://distro.ibiblio.org/pub/linux/distributions/smeserver/mirrorlist/smedev-8
    Name=SME Server - dev
    Visible=no
    status=disabled
smeextras=repository
    EnableGroups=yes
    GPGCheck=yes
    MirrorList=http://distro.ibiblio.org/pub/linux/distributions/smeserver/mirrorlist/smeextras-8
    Name=SME Server - extras
    Visible=yes
    status=enabled
smeos=repository
    EnableGroups=yes
    GPGCheck=yes
    MirrorList=http://distro.ibiblio.org/pub/linux/distributions/smeserver/mirrorlist/smeos-8
    Name=SME Server - os
    Visible=yes
    status=enabled
smetest=repository
    EnableGroups=yes
    GPGCheck=yes
    MirrorList=http://distro.ibiblio.org/pub/linux/distributions/smeserver/mirrorlist/smetest-8
    Name=SME Server - test
    Visible=no
    status=disabled
smeupdates=repository
    EnableGroups=yes
    GPGCheck=yes
    MirrorList=http://distro.ibiblio.org/pub/linux/distributions/smeserver/mirrorlist/smeupdates-8
    Name=SME Server - updates
    Visible=yes
    status=disabled
smeupdates-local=repository
    BaseURL=http://192.168.11.2/repos/releases/8/smeupdates/$basearch
    EnableGroups=yes
    GPGCheck=yes
    Name=SME Server - updates (local)
    Visible=yes
    status=enabled
smeupdates-testing=repository
    EnableGroups=yes
    GPGCheck=yes
    MirrorList=http://distro.ibiblio.org/pub/linux/distributions/smeserver/mirrorlist/smeupdates-testing-8
    Name=SME Server - updates testing
    Visible=yes
    status=disabled
updates=repository
    EnableGroups=no
    Exclude=initscripts,libgsf
    GPGCheck=yes
    MirrorList=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates
    Name=CentOS - updates
    Visible=yes
    status=enabled
[root@dell ~]#


I can't see how it can possibly work without pointing the BaseURL of all the sme repositories to my local repos on the local network
http://192.168.11.2/repos/releases/8/....   and disabling the other entries to repositories on the internet that will otherwise just cause a long time out?  I might give this a try ...

Offline mealogav

  • *
  • 27
  • +0/-0
Re: Installing Contribs in sme without internet access
« Reply #29 on: July 11, 2013, 01:18:12 AM »
Ok  :-)

I've used "/sbin/e-smith/db yum_repositories" commands to change my repositories as follows.  Essentially smexxx repos have a BaseURL pointing to my local copy of that repository.  All others are disabled (just to speed it up otherwise you seem to get a long timeout for each repos that can't be reached via the MirrorList). 
Not forgeting "signal-event yum-modify" (which of course I did forget first time!)

And it's worked - sort off.  It has certainly updated loads of packages (250MB ).  But at the weekend I updated a new install from the current iso and I think there was something like 340MB of updates.
I think that for a complete update of a "standard" install with no contribs or extras, both the base and addons repositories are required - but hey I'm no expert.

db yum_repositories show  now outputs this ....  (and I don't think the update has changed it back to the defaults - but this is yet to be checked properly).  After checking properly I notice the post update process has reintroduced the MirrorList properties.  These should be removed before trying another update or contrib install.  See later posts for more info.

[root@dell ~]# /sbin/e-smith/db yum_repositories show
addons=repository
    EnableGroups=no
    GPGCheck=yes
    MirrorList=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=addons
    Name=CentOS - addons
    Visible=no
    status=disabled
base=repository
    EnableGroups=no
    Exclude=initscripts,libgsf
    GPGCheck=yes
    MirrorList=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os
    Name=CentOS - os
    Visible=yes
    status=disabled
centosplus=repository
    EnableGroups=no
    GPGCheck=yes
    MirrorList=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplus
    Name=CentOS - centosplus
    Visible=no
    status=disabled
contrib=repository
    EnableGroups=no
    GPGCheck=yes
    MirrorList=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=contrib
    Name=CentOS - contrib
    Visible=no
    status=disabled
extras=repository
    EnableGroups=no
    GPGCheck=yes
    MirrorList=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras
    Name=CentOS - extras
    Visible=no
    status=disabled
smeaddons=repository
    BaseURL=http://192.168.11.2/repos/releases/8/smeaddons/i386
    EnableGroups=yes
    GPGCheck=yes
    MirrorList=http://mirrorlist.contribs.org/mirrorlist/smeaddons-8
    Name=SME Server - addons
    Visible=yes
    status=enabled
smecontribs=repository
    BaseURL=http://192.168.11.2/repos/releases/8/smecontribs/i386
    EnableGroups=yes
    GPGCheck=yes
    MirrorList=http://mirrorlist.contribs.org/mirrorlist/smecontribs-8
    Name=SME Server - contribs
    Visible=yes
    status=enabled
smedev=repository
    EnableGroups=yes
    GPGCheck=yes
    MirrorList=http://mirrorlist.contribs.org/mirrorlist/smedev-8
    Name=SME Server - dev
    Visible=no
    status=disabled
smeextras=repository
    BaseURL=http://192.168.11.2/repos/releases/8/smeextras/i386
    EnableGroups=yes
    GPGCheck=yes
    MirrorList=http://mirrorlist.contribs.org/mirrorlist/smeextras-8
    Name=SME Server - extras
    Visible=yes
    status=enabled
smeos=repository
    BaseURL=http://192.168.11.2/repos/releases/8/smeos/i386
    EnableGroups=yes
    GPGCheck=yes
    MirrorList=http://mirrorlist.contribs.org/mirrorlist/smeos-8
    Name=SME Server - os
    Visible=yes
    status=enabled
smetest=repository
    EnableGroups=yes
    GPGCheck=yes
    MirrorList=http://mirrorlist.contribs.org/mirrorlist/smetest-8
    Name=SME Server - test
    Visible=no
    status=disabled
smeupdates=repository
    BaseURL=http://192.168.11.2/repos/releases/8/smeupdates/i386
    EnableGroups=yes
    GPGCheck=yes
    MirrorList=http://mirrorlist.contribs.org/mirrorlist/smeupdates-8
    Name=SME Server - updates
    Visible=yes
    status=enabled
smeupdates-local=repository
    BaseURL=http://192.168.11.2/repos/releases/8/smeupdates/$basearch
    EnableGroups=yes
    GPGCheck=yes
    Name=SME Server - updates (local)
    Visible=yes
    status=disabled
smeupdates-testing=repository
    EnableGroups=yes
    GPGCheck=yes
    MirrorList=http://mirrorlist.contribs.org/mirrorlist/smeupdates-testing-8
    Name=SME Server - updates testing
    Visible=yes
    status=disabled
updates=repository
    EnableGroups=no
    Exclude=initscripts,libgsf
    GPGCheck=yes
    MirrorList=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates
    Name=CentOS - updates
    Visible=yes
    status=disabled
[root@dell ~]#
« Last Edit: July 11, 2013, 11:41:20 AM by mealogav »