Tag Archives: migration

Posted on
by

elgg_logo1 Here’s the scenario; you’re a developer and you’ve been asked to do some work on an existing Elgg site, or you’ve built an Elgg site with some complex plugin interdependencies that you need to copy on to a live site.

In both cases, this primarily involves copying the source code and Elgg database from one site to another, here’s how…

Source code and database

  1. Install the source code for your project; scp it from the other site, git clone –recursive, whatever…
  2. On the site you’re copying, take a dump of the database. You can look in your engine/settings.php for the database username and password:

    mysqldump -u your-db-user -p elgg_database > database-dump.sql

  3. Copy this file onto your new host.
  4. Create a new database and install the Elgg database into it, in the mysql client do the following:

    create database new_elgg_database;
    grant all on new_elgg_database.* to `db_username`@`localhost` identified by 'db-password';
    use new_elgg_database;
    source /path/to/database-dump.sql

  5. You should now have a local copy of the elgg database installed, but in order for it to work you need to change a few paths. Firstly, alter your dataroot and site location details in your prefix_datalists table:

    update elggdatalists set value="/path/to/elgg/" where name="path";
    update elggdatalists set value="/path/to/dataroot/" where name="dataroot";

    Don’t forget the trailing slash on the paths!

  6. Next, you need to update the site url in the site object stored in the prefix_sites_entity table. For the vast majority of people (who only have one site object) this will be straightforward, for others, you might have to use a slightly different query in order to get all sites working as expected.

    update elggsites_entity set url="http://localhost/path/to/site/";

    Again, don’t forget the trailing slash on the URL!

  7. Finally, alter your copy of engine/settings.php to reflect your new database details.

When I view my site, all the CSS is broken!

This is almost certainly a mod-rewrite problem.

  • Firstly, check that it’s installed and enabled, and that overrides are enabled for your site URL (common problem if installing into ~/public_html).
  • Next, make sure that your RewriteBase is configured. If you’re installing into a subdirectory on a domain (e.g. http://localhost/~marcus/elgg/) you’ll need to set the RewriteBase in your .htaccess file accordingly, in the case of my example, RewriteBase /~marcus/elgg/

Files

The above should get you up and running with a usable site for testing, however if you want to fully migrate a site, you’ll also need to copy the data directory across.

  1. Using rsync or similar copy the complete data directory from your old site’s data directory to the new.
  2. Ensure that the directory, subdirectories and files are read and writeable by your web server’s user.
  3. Flush the caches. This is important since Elgg caches the locations of template files and other data in the data directory, which obviously will cause issues if you copy a cache file from another location! If the admin panel has become unavailable at this point, deleting the system_cache directory from dataroot by hand will often restore it.

Happy hacking!

Posted on
by

So, I recently performed some long overdue upgrades on my work computers, which involved performing a complete from scratch rebuild of the house server.

This involved, among other things, moving the OpenLDAP directory in which I store, among other things, my email address book. Easier said than done.

Previously, when I had done this, it was a simple matter of installing slapd, copying /etc/ldap and /var/lib/ldap and restarting slapd. This time proved more tricky; slapd initially spat out errors complaining that the database was corrupt. Check out https://www.sapphire.net/mss/incident-response-services/ for more information on the benefits of improving your cybersecurity at work.

Fixing the corrupt database could be done in one of two ways: 1) deleting /var/lib/ldap/DB_CONFIG, forcing slapd to recover the database, or 2) running db_recover directly. After this was done, slapcat displayed the directory as expected, and slapd restarted, however any queries on the database – via phpldapadmin, ldapsearch or the email address book – caused the answering slapd process to lock with no error. Each subsequent query would cause another slapd thread to lock, eventually bringing down the whole ldap server.

Hmm…

I’m not entirely sure what was going on, perhaps it was a bug in my particular version of slapd, or perhaps it was the fact that I was moving from 32bit to 64bit, who knows. I eventually found a solution, and I write this down primarily as a note for myself.

The workaround

    1. Install slapd, copy the config and db over, then run a db recover on the database:

      db_recover -v -h /var/lib/ldap

    2. You should now be able to list the directory with slapcat, so dump it to a file:

      slapcat > /tmp/directory.ldif

    3. Next, we need to nuke the database and force slapd to create a fresh blank database. There’s probably an easy way of doing this, but the quickest way I found was to delete /var/lib/ldap/* and reinstall slapd:

      cd /var/lib/ldap; rm *; apt-get remove slapd; apt-get install slapd.

The apt-get remove will remove the program but by default it’ll leave your configuration intact, if it doesn’t you might need to copy your config back as well.

  1. Next, stop slapd and import your ldif file.

    /etc/init.d/slapd stop; slapadd -l /tmp/directory.ldif; /etc/init.d/slapd start

If you had the same problem as me, your OpenLDAP directory should now be back in place, imported into the new database. For me, my address book now functions correctly and phpldapadmin no longer hangs.

There are probably easier ways, but I was tired. Let me have your thoughts in the comments below!

Posted on
by

Gitosis is a GIT server system which, using ssh, lets you run a central git repository in much the same way as github does. This let you manage multiple developers easier, as well as providing a convenient place to access repositories while out and about, and for deployment.

Unfortunately, gitosis is no longer maintained, and has been removed from more recent versions of the major linux distributions. This was preventing me from performing some much needed server upgrades, so it was therefore necessary to migrate to another bit of software.

Gitolite is the recommended replacement for Gitosis, and acts as a drop in replacement. Perform the migration right, and you’re users will never notice that you did anything at all.

So, in hopes that this may be useful to someone, here’s how I migrated my gitosis server over.

The initial setup

The initial server configuration was as follows:

  • Debian server
  • Gitosis installed as user “git”

My goal was to replace the gitosis server, still on the GIT user, so my users would not need to modify any of the remote repository paths in any checked out repositories.

Migration

Start off by taking a backup, just incase this goes horribly wrong, then…

  1. Belts and braces, get rid of the old gitosis update hooks and prevent any new sessions by removing the authorized_keys file: mv git/repositories/gitosis-admin/hooks/post-update git/repositories/gitosis-admin/hooks/post-update.old; mv git/.ssh/authorized_keys git/.ssh/authorized_keys.old
  2. Move the old gitosis home directory out of the way: mv git git_old
  3. Install gitolite: apt-get install gitolite
  4. I then needed to reconfigure gitolite so it used the same user id as the previous gitosis install: dpkg-reconfigure gitolite
  5. Copy your old repositories to your newly created git directory: cp -a git_old/repositories git/
  6. Gitolite had trouble using my existing public ssh keys for the admin account, probably because they were already used as login keys, or perhaps because they were in the foo@bar.pub format. Either way, the simplest thing was to generate an admin key specifically for gitolite administration.
    1. Generate a new key, making sure you have at least one “.” after the “@”, so that the key looks like an email address: ssh-keygen -t rsa -C "gitoliteadmin@myserver.local" -f gitoliteadmin@myserver.local
    2. Make sure root, or whoever is going to admin your gitolite repo has a copy of these keys, as you’ll need them to make any configuration changes. You can simplify this somewhat by making a host alias for the gitolite admin user in the ~/.ssh/config file

      Host gitolite-admin
          HostName myserver.local
          User git
          IdentityFile ~/.ssh/gitoliteadmin@myserver.local

  7. On the server, change to the git user: su git
  8. Then initialise the gitolite repository, passing the location of your newly created admin key: gl-setup /path/to/gitoliteadmin@myserver.local
  9. Clone the gitolite-admin repository, note the use of the gitolite-admin host repository: git clone git@gitolite-admin
  10. Convert your gitosis settings file using gl-conf-convert, which if you’re running this on the server, can be found in /usr/share/gitolite. This script can be run in isolation, so it’s ok to copy it about if you need to run this on a different machine: /path/to/gl-conf-convert < /path/to/gitosis-admin/gitosis.conf >> /path/to/gitolite-admin/conf/gitolite.conf
  11. Now, check your gitolite.conf for errors, and if ok commit and push your changes. Since I had a number of keys in the format of user@machine, I had to change the occurrence of those users in the file to just the username before the “@” character. E.g. foo@machine becomes just foo
  12. Things should now be working on gitolite. You can verify that gitolite rather than gitosis is fielding your requests using ssh: ssh git@myserver.local info, you should see a list of the repositories on the server that your user has access to.

All being well, your migration over to gitolite should now be complete, and remotes in any existing clones of repositories on the server should still function.

Hope this helps!