Gandi Kitchen - Hosting

Mirror server and CentOS

aegiap — Fri, 30 Dec 2011 11:27:00 +0100

Information about the CentOS repository on Gandi mirror server.

Local CentOS mirror

In order to reconfigure our CentOS mirror repository the way CentOS use it, the default version '6' which is currently pointing to the 6.1 version will be switch to 6.2 version in the beginning of january (hopefully january the 5th). With this configuration change, our local CentOS repository will be compliant with upstream mirror and customer will be able to easily follow the version of the 6 branch with simple update operations.

CentOS 6 image

Following the release of the 6.2 version of CentOS, both 32 bits and 64 bits OS images on Paris and Baltimore datacenters are now updated with the new version.

Local mirror URL

The URL of the official mirror server which contains distributions tree is http://mirrors.gandi.net. The index page is listing all the distributions and systems availables.

With Debian, for example, the URL to add to /etc/apt/sources.list is :

deb http://mirrors.gandi.net/debian squeeze main experimental non-free

Older URL like http://centos.mirror.gandi.net/.../ or http://debian.mirror.gandi.net/.../ are now obsolete and will be removed soon. They were setup during the beta step of the IAAS hosting.

The mirror server is available for virtual server on IAAS hosting and is not reachable for servers outside Gandi network.

What's Happening at the Cloud Expo in Santa Clara?

Thomas — Mon, 14 Nov 2011 11:03:00 -0800

Gandi's Thomas Stocking went to the Cloud Expo 2011 West in Santa Clara this week, and found that marketing is alive and well in the Cloudsphere. Despite the fluff, there were some interesting ideas getting discussed, and companies developing strategies to leverage the evolving cloudscape...
What's Happening at the Cloud Expo?

Attending Cloud Expo 2011 West in Santa Clara requires a lot of stamina. The conference is a grueling 4 days long, and has a whopping 160 sessions and lectures. Quantity, however, does not always equal quality, and I was a bit underwhelmed by much of what was presented. Enough marketing fluff! I come to conferences to get to the core technical truths. Some interesting ideas were being discussed, of course, but weeding them out of the pitch and jargon was more challenging than I like to put up with. If I want a challenging search, I’ll go geocacheing!

Ok, so what were the interesting ideas? Well, for one thing, Open Source (capitalization is mine) is clearly in the forefront of innovation in the cloud space. I spoke with Krishnan Subramanian who contended in his talk that Stallman got it wrong and that cloud technology is not evil, or inimical to Open Source, and in fact that Open Source is a necessary counterpoint to the imminent consolidation of cloud vendors for the enterprise space. I thought it was interesting that Krishnan is admitting that companies (Rackspace for instance, with the release of OpenStack) are going to be developing Open Source, rather than relying on individual users to find and fix bugs. This means the Open Source software development model will change from the “with enough eyeballs, all bugs are shallow” to a company-sponsored community feedback model. That’s not the “free software” model Stallman is talking about, but it’s still a viable one, or at least innovators had better hope it is. Greenqloud CEO Eirikur Hrafnsson was there, and we talked a bit about the relative merits of CloudStack and OpenStack, which Greenqloud is enhancing for use in it's environmentally optimized data centers.

Speaking of community development, it seems like there is a fair amount of thought going into this as well, with cloud computing offering some unique opportunities to leverage data that users input, or at least the way they use the data. In the traditional model of client-deployed software, the poor lonesome developers really had no easy way to see what the users were actually doing with their software, whereas if the users access a hosted solution, common use cases become obvious in the usage patterns. A SaaS vendor who can detect and capture this data can quickly adjust a difficult workflow, for example. Dell Boomi, a hosted solution for mapping data exchanges and workflows between enterprise applications, leverages the fact that their SaaS is actually a shared model, and so the data for such things as field mappings that users input can be used to “suggest” field mappings based on work done by other users. Cloud vendors could do the same but would need to do more to harvest that data across multiple instances. It’s easy to see how this helps developers and product managers do more to select popular features for focused development. Gandi does that: we choose the packages for the Simple Hosting offering based on what our cloud hosting customers are choosing most often.

On the show floor there were many examples of what I’ll term epiphytic businesses. Players like Teremark and PhoenixNap offer added value on top of VMware hosting technology. They pay VMware for that in license fees, of course. Terremark scored big by getting GSA listing and tapping into the billions pledged by the federal government to Cloud Computing, which is probably one good reason why Verizon Business bought them. Lest you be an American taxpayer and think this is yet another boondoggle, the case is really good for consolidation of many, many redundant federal data centers.

Apparently the hunt for differentiation on the basic value proposition of cloud computing is still going on, and some vendors are striking pay dirt. Other epiphytic businesses tackle areas that are key to cloud adoption, but that the main cloud technology providers don’t really want to mess with, like enhanced security, or end user support. CloudPassage has a way of dealing with security using agents deployed to instances. Perspecsys has a middleman security scanner that aims to protect data in the cloud from sloppy or compromised user systems. Appriver provides desperately needed end-user support and front-end sales to Microsoft’s nacent Office 360 packages.

Some of these business are dependent on the “mother” technology vendor, and others, trying to defend themselves from this risk are building interfaces to all the providers they judge to have staying power. One such is a cloud management platform called Abiquo that gives you a nice gui (of course) as well as an API on top of “all the major cloud vendors” offerings. This app lets you build, migrate, and monitor virtual data centers with your installed private and public clouds (where have I heard this before?).

Good ideas are being bought: Gluster’s cloud-based storage clustering solution was acquired by Redhat, and this made many sit up and take notice, I’m sure. Redhat has also entered the PaaS fray with OpenShift, which will presumably allow them to take advantage of the great work done to manage linux packages and, of course the JBOSS Java EE framework. Perhaps Gluster’s sauce will make that meat more savory?

What’s Still Missing?

Cloudbursting remains elusive, that is, the true, automated ability to augment private cloud capacity with public cloud resources to achieve higher scale for short periods.
Standards are lacking. APIs abound, like Amazon’s API for EC2, and despite efforts of well-intentioned people in places like the Open Data Center Alliance to come up with shared and complete models for us to work with, it will be a while before we settle on one (or two, or three) standards.

What's Next for Gandi in the US?

We are going to keep developing out cloud hosting offering at Gandi SAS, and Gandi US is going to make sure the American customers get excellent support. We will be expanding our US presence, so look for us in your town! See you at the LISA conference in Boston in December! Happy computing.

Storage Migration

Leland Vandervort — Thu, 05 May 2011 15:09:00 +0200

For the past few months perhaps you might have made use of the servers in the US. The changes in the storage technology was one of the strong points in the hosting infrastructure. Prior to this, we had to adapt the infrastructure so that it could understand "n" datacenters. The implementation of this new storage platform was not as complicated as it seems since it is completely independent to the architecture in France. With a new datacenter, it was therefore quite a trivial matter to build from scratch, and all of the new servers in the US made use of this new platform from day one.

On the French side, an inevitable migration was required in order to arrive at a standardised platform to utilise the new functionalities. Here, however, the problem is different and a little more complicated, with the coexistence of two different storage solutions. In reality, we were confronted with a number of challenges so that the machines hosting the servers could happily play ball with two different storage platforms at the same time. The opening of our US datacenter was already a few months ago, and so a large proportion of our efforts have been dedicated to this migration. This is, of course, proceeding and certainly takes [a lot of] time, and will soon be available to all of our customers. We will thus be in a position to run the two storage platforms together in order to perform the migrations efficiently.

Once active, all new disk creations will take place on the new storage platform. At this point we will enter the migration phase, and specifically the phase which directly impacts you since it also means the migration of your disks.

There are a number of ways to perform the migration:

Create a new disk:

The creation of this new disk will be on the new platform.

Next, simply attach this new disk to your server and copy the relevant data to the new disk. This would be a good occasion to do some housekeeping and get rid of any old data that you no longer need. Such commands as 'cp' or 'rsync' would do the job nicely.

Create a new disk from the image of an existing one:

This function has been available through the API for several months: disk.create_from(apikey, disk_spec, src_disk_id) -- see the API documentation for more details at http://doc.rpc.gandi.net/hosting/reference.html#disk.create_from

The next Gandi Website update will include the ability to create a new disk from and existing one to make life easier for you if you are not a user of our API. This method, nevertheless, requires either that the server that has the original disk attached be stopped, or that the disk be detached from the server.

Please note that the time take to make the copy will be directly dependent upon the size of the disk, so you should have some patience and/or a good coffee break, if you decide to employ this method on a comparatively large disk.

If the new disk is to be a system disk, then you will need to define the disk as a boot-disk either via the web interface ("Boot Disk" in the server details) or using the API : vm.disk_attach(apikey, vm_id, disk_id {'position' : 0} ) ( see http://doc.rpc.gandi.net/hosting/reference.html#vm.disk_attach )

This migration will enable you, among other things, to make use of the new storage features such as snapshots, resizing, and rapid-copy, which will be available in the coming weeks. Aside from the platform which is seeing new features every month, we hope to soon be able to talk to you about these new functions which may change the way you use your servers.

New storage infrastructure

William — Mon, 24 Jan 2011 23:42:00 +0100

Our new storage infrastructure is being deployed.

Disk Management is now handled through virtualization... and we are excited to share it with you!

As you will be seeing, the features are very interesting and we will be integrating them over time in your management interface as well as in the new API.

Improvements

Besides the disks, still using RAID for redundancy and a write cache resistant to a power failure, the storage servers are now organized in clusters: in case of controller failure, a standby controller is ready to take over the management of disks.

The storage capacity has been improved as well as disk I/O. Moreover, you will be soon able to create larger disks.

Creating, copying and resizing disks will be faster and more automated.

Snapshot

This mechanism enables you to create a snapshot of your disk that is either in use or detached. The snapshot can be used to achieve a consistent backup of your disk. Within a few weeks, a snapshot on demand will be available. We are also working on periodic management of snapshots. For example the ability to create a snapshot every day and only the last seven.

Copying disk

Thanks to the snapshots, it will be possible to create a fast and live copy of your disk, even with activity on it. This feature will facilitate fast deployment (clone) of servers.

Rollback

From a snapshot you will be able to create a disk, getting back your lost data or apply a rollback operation on your disk.

Automatic backup drive

A backup system, disk to disk, will be available in the future; we are still working on the specifications. For most of you, the snapshot mechanism will probably fulfill your needs.

Disks and Partitioning

Gandi disks are logical drives, meaning that we provide an access to disk already optimized, secured; it is easy to create new one, resize or delete them.

The partitions, or RAID algorithms are for your virtual machine, at best an unnecessary overhead, and at worst a reorganization of disk accesses that can result in performance degradation (see below). In addition, you can deprive yourself of convenient features: a snapshot made on a disk that is part of a RAID has no interest, resizing a partitioned drive (our old system) is complex even dangerous if not made carefuly.

Like with SSD and new hard disks, blocks are not 512 bytes long anymore but 4KB. If you are using partitions, access to your data may not be aligned, and performance divided by three.

Partitions are helpful for physical disks. This complicates the management of your disks for nothing: consolidate your data on a single disk or create multiple data disks is the right solution.

The disks created by Gandi will no longer use partitions. Currently only the system disks have a default partition (system and swap): it will disappear. The system disk will be managed like any other disk, which will facilitate management. To keep the same level of service, we will provide a temporary disk for swap. To remain consistent with the current configurations, we will emulate virtual partitions to present the system disk and swap: the system will be seen as xvda1, the swap as xvda2, like on the current system but it will be two different disks.

Note that the new storage infrastructure is available form the beginning in our Baltimore data center; features will be along the time.

New kernels available for your server.

aegiap — Mon, 17 Jan 2011 16:32:00 +0100

The kernel list on Gandi hosting grows. You have now the choice to use a 2.6.36 or a 2.6.32 with the grsecurity patch.

Gandi hosting allow customer to choose from a pool of patched and built kernels from Gandi team. Three new kernels have been added to the available kernel list :

2.6.36 in x86_32 and x86_64

The Linux kernel is in active development : new versions containing features and bugfixes are published on a regular basis. We have prepared new build in 32 bits and 64 bits of the 2.6.36 kernel with D.Kipper patch about additional memory managment. Moreover the feature of resizing an online disk attached to a server is now merged into the kernel. The feature will arrive soon in the hostinig interface.

2.6.32 in x86_64 with grsecurity patch

Previous kernels allowed customers to setup security solution in the server system (thanks to selinux for example). When choosing this new 2.6.32 kernel you should be able to setup an advanced security policy thanks to grsecurity in version 2.1.14-201005151340. The kernel allow disk replication thanks to DRBD in version 8.3.8 which was not yet merged in the upstream source tree (integrated since the 2.6.33).

You can check the changelog of Gandi kernels and the process to update kernel modules according to your selected kernel.

Gandi Hosting : US and France Datacenters FAQ

Leland Vandervort — Thu, 23 Dec 2010 14:33:00 +0100

With the opening of our Baltimore datacenter, we have decided to provide a quick FAQ to respond to a few of the commonly asked questions about the hosting product and how it will work with the two datacenters, as well as some of the other features that we are putting in place.

If you have any questions not addressed here, then please feel free to let us know!

Q: Can I migrate my server automatically from Paris to Baltimore, or vice-versa?
A: No; there is no automatic solution to migrate a virtual server between the Paris and Baltimore datacenters due to technical limitations (IP addressing, platform differences, etc.)

Q: Okay, but I really want to migrate my server... how can I do it ?

A: If you are really sure, then there are a couple of ways to do it.
Firstly you will need to create a new server through the admin interface (or public API). You will then need to transfer the data from the old server to the new one.

For Gandi AI, you will need to save the source files for your website and export the database, for example, with PHPMyAdmin, and then import these on the new server.

If your server is running in expert mode, it is possible to transfer these files directly over SSH. Another method, again in expert mode, involves creating a new server with a data disk in the target datacenter, and transfer the system disk from the old server to the newly attached data disk of the new server (for example, with 'dd' but please note that this may take some time).

Finally you can then change the disk type from data to system via the admin interface in the advanced configuration for the disk, shut down the new VM, remove the old system disk and attach this new image as the new system disk, then start the VM.

There are a few methods to achieve this:

With 'dd/gzip':

dd iflag=direct bs=8k if=/dev/xvdYX | gzip -9 | ssh baltimore-server "gzip -d | dd bs=8k of=/dev/xvdYx"

With 'mbuffer':

1. First start the target on the Baltimore server:

mbuffer -q -s 128k -m 8M -I <baltimore_server>:<port> | gzip -d | dd bs=8k of=/dev/xvdYX

2. Start the sender on the Paris server:

dd iflag=direct bs=8k if=/dev/xvdYX | gzip -9 | mbuffer -q -s 128k -m 2M -0 <baltimore_server>:<port>

With 'rsync':
You can simply copy the files that you need.

    rsync -arvz src baltimore_server:/dst

Q: Can I detach the IP address from my Paris server and attach it to my Baltimore server?

A: No; the IP address allocations and the server provisioning systems for the two datacenters are completely independent.

Q: Can I attach a disk in the Paris datacenter onto my server in the Baltimore, for example, to make an off-site backup?

A: No; for the same reasons that an automatic migration between datacenters is not possible, the two platforms are completely independent.

Q: I see that Gandi will be deploying Anycast DNS servers, will this allow me to do geolocalization for my servers?

A: No; the anycast DNS servers simply give a physical presence for the Gandi DNS servers in multiple locations so that DNS lookup requests can be answered by the closest server. The DNS service does not provide any geolocalization capability.

Q: Can I have a server in Paris and another one in Baltimore and use anycast to allow the closest server to serve the content?

A: No, because the hosting platforms are independent of each other and use separate IP allocations.

Q: I have my own ASN and assigned /24 for Anycast purposes, can I use a Gandi VPS to host my anycast service, assuming that I use other providers to expand the geographic and network scope?

A: It is technically possible to do this under certain conditions; We have already tested such a solution, but it is entirely bespoke and has certain limitations concerning how your own IP block can be used on your VM. You will need to discuss your requirements with our technical team to determine if we can accommodate your requirement.

Q: Why when I traceroute from my Paris to my Baltimore server, do I have a 92ms latency all of a sudden?

A: Latency is defined as the one-way path delay between two points on a network.; What you see in a traceroute is the round-trip delay for the hop in question (with a special exception for hops contained within an MPLS label-switched-path -- see the next question)

Transatlantic latency plus the physical fiber cable distances incur a one-way path delay of roughly 45ms (plus or minus a little). While it is true that at the speed of light the delay to cross the Atlantic is only in the order of 28ms in a straight line, you need to add in the factors of ocean depth, and then the land-based cable distances between the two end locations. This translates to a round-trip delay of at least 90ms, thus this is normal.

(This is a simplified explanation, as there are actually a number of factors that determine the delay observed, and we would be happy to discuss these with those who are really interested in the nitty gritty details...)

Q: When I traceroute from my Paris to my Baltimore server, I reach a router which is still in Paris but shows a 92ms round-trip time, even though the hop just before it is only 1ms. Doesn't this point to a problem on your network?

A: Our network is running MPLS (MultiProtocol Label Switching) in the core, with Traffic Engineering (TE) enabled. As such, when a packet crosses the network and enters a known end-to-end pathway between two endpoints (known as a Label-Switched-Path, or LSP), the IP packets remain within that pathway until they egress at the remote end if there is an existing TE tunnel along that path.

As a result, the intermediate routers within the LSP will still respond to the traceroute with the ICMP TTL-Exceeded packets as normal, but these packets are forwarded along the full path and back before being sent back to the originating source of the traceroute.

In consequence, the round-trip-time that you see for the Paris-side router in this instance is actually the round-trip-time for the full MPLS path, plus the time between the source and the router itself. Again, this is perfectly normal behavior.

Q: Why do you run MPLS on your network?

A: MPLS allows for a number of different solutions, not least of which is providing traffic engineering and flow control based on certain criteria. We can specify certain traffic based on its characteristics to follow a given MPLS path under normal circumstances, while still allowing for alternative paths in case of link failures.

We can also make use of other features of MPLS to enable layer2 and layer3 VPN connectivity across different portions of the network for some of the Gandi services, whilst keeping the size of the routing tables as low as possible. (The DNS servers being deployed in anycast use some of the features provided by MPLS across the core of the network, for example).

Q: I have a server in Baltimore and one in Paris. Can I have a second interface each with a private VLAN to connect the two servers with a back-end private network?

A: We are working on a private VLAN solution to be deployed later in 2011 and we are looking at ways to provide private VLAN connectivity for customers between their servers in different datacenters.

Q: I'm not convinced! This is a new datacenter, and I am not running any services yet, but even between my server and its next hop router I get weird and inconsistent ping results. Why?

A: This is due to hardware buffer sizes on physical ethernet interfaces. In order to transmit data physically "onto the wire" the interface needs to fill the buffer. If there is little or no traffic, small amounts of data will be "stored" in the buffer until there is enough data to transmit. This will result in false delay/round-trip and variance/jitter readings. Under normal data usage, this phenomenon is much less evident because the buffers will fill more quickly. It is, nevertheless, completely normal behavior for any network device or server NIC.

We hope that these help answer some of the questions that you have expressed. We will additionally be putting this FAQ along with any new questions and answers over time on Gandi's online knowledge center (http://wiki.gandi.net), so please check back from time to time for updates!

How to Create a System Images for Your Server

Leland Vandervort — Tue, 16 Nov 2010 16:46:00 +0100

There are many reasons to create a system image for your servers: to build a custom system with your preferred applications pre-installed, to create an image of a game server that can be easily deployed, to simply duplicate a custom server, or simply to backup one's system...

The procedure is relatively simple and can be performed by anybody, as long as you pay careful attention to the detail.

Create a Data Disk

You need to create a data disk via the disk creation interface of your hosting account. If you wish to make a copy of an existing disk, the data disk must be of sufficient capacity.

Simply attach your new disk to the target server ; the server that contains the virtual disk to copy, or the server used to perform the base installation.

Create a System Image

By making a copy of the data from a virtual disk.

Warning: You must have sufficient space on the destination disk.

Copy the data from the source disk to the destination with the 'tar' command:

tar cC /srv/disk1 . | tar xC /srv/disk2

If /srv/disk1 is the directory where the source disk is mounted and /srv/disk2 is the mount point of the destination disk. Using 'tar' will only copy the data and not empty blocks, so the operation is relatively quick.

Warning: if you want to copy the local system disk, you must add a few exclusions:

tar
--exclude=/proc --exclude=/sys --exclude=/srv/disk2 -c \ | tar xC /srv/disk2

By installing a base system

Preamble

Your new disk attached to a server is ready for use. You can partition your disk to prepare a swap partition for the image, but we advise that you do not create partitions and to create the file system directly on the disk.

In this way you have the advantage of flexibility if you intend to resize the disk later (no need to calculate the partition table), whilst being better adapted to the block storage on our physical filers, and to be able to decide later whether or not to use a swap file.

In the near future, all of the OS images provided by Gandi will be without partitions, and the Gandi hosting platform will automatically provide additional swap space.

Meanwhile, the virtual servers will still see the first disk as "xvda1" and the swap disk as "xvda2".

Whether or not you decide to partition, the Gandi hosting platform will be able to start a system disk with your image as source by detecting the disk boot sector and adapting the kernel boot options. Gandi also provide a server containing copies of the distributions for which we provide system images. You can use this server to speed up your installation : mirrors.gandi.net

Preparation

Initially, verify that the virtual disk is not already mounted (for example in /proc/mounts). Unmount the disk if it is mounted. Partition the disk if required and then format it.

We recommend that you format the disk directly and use the ext4 file system. If, for example, your disk is xvdc : mkfs.ext4 -j -m0 /dev/xvdc and then mount the disk to a directory of your system : mount -o rw /dev/xvdc /var/tmp

Bootstrap installation for distributions using .deb packages

The base system installation can be accomplished with debootstrap in a specific directory. Once the installation is completed and a few modifications are made, the disk will contain a bootable and fully functional GNU/Linux system. The system will have a basic set of applications and you will need to adapt the system to your requirements by installing any required applications with apt-get and configuring the locales.

We highly recommend the installation of a system with the amd64/x86_64 architecture, as this will be faster on the Gandi platform. When it comes to selecting the kernel, you should choose the appropriate x86_64 kernel (2.6.32-3831, for example).

An example installation for an Ubuntu Maverick 10.10 distribution on a previously prepared and mounted disk:

debootstrap
--arch=amd64 --verbose --components=main,universe,multiverse \

--include=openssh-server,openssh-client
maverick /var/tmp \

http://mirrors.gandi.net/ubuntu/

Should debootstrap happen to complain that no configuration file for maverick is accessible, you only need create a link in /usr/share/debootstrap/scripts from lucid to maverick. The following step will modify the file containing the source package media locations. Edit the file /etc/apt/sources.list in the directory by adding the distribution media and the Gandi package media.

Example :

# cat
/var/tmp/etc/apt/sources.list

deb
http://mirrors.gandi.net/ubuntu maverick main universe multiverse

deb
http://mirrors.gandi.net/ubuntu maverick-security main universe multiverse

deb
http://mirrors.gandi.net/ubuntu maverick-updates main universe multiverse

deb
http://mirrors.gandi.net/gandi/ubuntu maverick main

Once the debootstrap command and the configuration has been completed, you will be able to access the newly installed system via chroot to the directory.

For example: chroot /var/tmp

In this way, you can refresh the package media with apt-get update and install the various applications and packages that you may require. Beware, however, that the installation of packages in a chroot require a few corrections. Notably, you will need to mount /proc and /dev/pts which are usually available in /etc/fstab : mount -a then complete the package configurations with dpkg --configure -a. Certain packages attempt to automatically start the daemons and servers as they are installed. You will have to stop these services to umount your image later.

In order to complete the installation of these packages, you will need to modify the postinst files installed in /var/lib/dpkg/info by commenting out the calls to invoke-rc.d or start, such as with procps : #start procps in procps.postinst or for rsyslog : #invoke-rc.d in rsyslog.posinst.

For recent Ubuntu distributions, you will need to add an entry for /dev in the /etc/fstab file: dev /dev tmpfs rw 0 0 in order to boot correctly.

Next, copy the files /etc/hosts, /etc/resolv.conf and /etc/fstab of the server to the directory containing the newly installed system (/var/tmp in our example). Change these by deleting the hostname of the current machine in /etc/hosts and adapting /etc/fstab to your new disk image.

We recommend installing the gandi-hosting-agent and gandi-hosting-vm packages available on the Gandi distribution media server. To do so, add the Gandi maintainer key as follows:

apt-key adv --keyserver keyserver.ubuntu.com --recv-keys D8EAC2F4DAFE3FA5

then :

apt-get install gandi-hosting-agent gandi-hosting-vm

The packages will install their required dependencies. For a description of the functions of these packages, please refer to the previous article in the Gandi Kitchen.

If your virtual server uses python version 2.6, you will need to install the package gandi-hosting-agent-py2.6 instead of gandi-hosting-agent.

Basic Installation for Distributions using .rpm

The installation of a distribution based on .rpm packages follows the same method as for a .deb based distribution, but using rinse or a specific application for the chosen distribution.

For example, using the same directory :

rinse --arch=amd64 --directory=/var/tmp --distribution=centos-5

According to your package management system, you will need to then add the package media locations in the configuration in order to install the necessary applications.

Please refer to the debootstrap / .deb base system installation for the remaining step : Gandi specific package installation, copy of system configuration file.

Other Ways to Install a Base System

Other distributions (often more specific) have other methods of installing the base system. Sometimes an image is already available, thus you need only mount it in 'loop' and copy the files from the image.

Taking the above example :

mkdir /srv/a

mount -o loop,ro /my/image/directory /srv/a

cp -raf /srv/a/ /var/tmp/

Yet a More Radical Method

If you have a source disk to copy, you could also copy the entire disk; Ensure that the destination disk that was just attached to the server is not mounted, and verify that the disk nameis not present in /proc/mounts.

Warning : this method only works where the destination disk is the same size as the source.

To create your system image and copy it to your new data disk, you will use the 'dd' command. You need to pay close attention to the syntax of the command.

Here is an example, which will be explained afterwards:

dd if=/dev/xvda of=/dev/xvdc conv=sync

The parameter if= refers to the source disk; of= refers to, of course, the destination disk. Any data on the destination disk will be completed erased. In order to distinguish between the disks, we recommend first using the 'df' command.

Note that the image will not be bootable. You will need to reconfigure the network settings of your server using DHCP before proceeding with the creation of the image. Copying using 'dd' takes significant time and resources as it copies byte-per-byte the source disk to the destination, without any distinction between a data block and an empty block. The Linux kernel for the Gandi hosting platform boots by default on the first partition ( root=/dev/xvda1 on the boot command line). If you copy an existing system disk, the partition table will be correct on the destination. Otherwise, you should ensure that the first partition of the destination disk contains the system files and is flagged as bootable. You can also additionally create a swap partition.

Requirements to Boot

In the case of a copy of an existing virtual disk, the source disk having already booted, you only need to check that the network configuration is using DHCP so that the produced images will be correctly bootable. In the case of an image created by a base system installation, there are a few items to verify and/or modify:

All of the services or modules related to physical elements of the server must be deactivated (such as, the service associated with the system clock - hwclock)

Some services at boot-time, such as ureadahead, console-setup, ondemand, plymouth, must be deactivated. Move away the init-scripts associated with these services. In an Ubuntu distribution, these scripts are found in /etc/init.

Add the gandi-hosting-agent and gandi-hosting-vm packages. The agent allows the installation of your server to complete. The scripts contained in the gandi-hosting-vm package allow the hosting platform to automatically and dynamically manage the server resources. For a system with python 2.6, you should install gandi-hosting-agent-py2.6 instead. For more details on these packages, please read the previous post on the Gandi Kitchen.

Delete the files related to ssh keys generated during the installation of the sshd package to avoid having the same key present on all of the servers generated from the same source image.

Verify that xinetd/inetd is started at boot -- if not, then activate it. The Gandi-agent is spawed via xinetd/inetd and will enable the final configuration of your server. The package gandi-hosting-agent installs its configuration file in /etc/gandi/agent.yml.

Verify that the permissions of your installation directory structure and the directories lib/ root/ and tmp/ of the installation directory. If in doubt, apply the same permissions to these directories as your existing server outside of the chroot.

chmod 0755 /var/tmp/lib

chmod 1777 /var/tmp/tmp

chmod 0750 /var/tmp/root

chmod 0755 /var/tmp/

Create the base files in /dev of your installation for the first boot stages, for example:

[ -e "$chroot"/dev/xvc0 ] || mknod "$chroot"/dev/xvc0 c 204 191

[ -e "$chroot"/dev/console ] || mknod "$chroot"/dev/console c 5 1

[ -e "$chroot"/dev/null ] || mknod "$chroot"/dev/null c 1 3

[ -e "$chroot"/dev/ptmx ] || mknod "$chroot"/dev/ptmx c 5 2

[ -e "$chroot"/dev/zero ] || mknod "$chroot"/dev/zero c 1 5

Add the necessary kernel modules corresponding to the kernel version in /lib/modules of your installation directory. The modules are available on mirrors.gandi.net/kernel/ (See the associated article on the wiki)

At the end of the configuration, if you are inside the chroot, leave it using 'exit' then unmount the various elements of your images in /proc/mounts. Kill any processes and daemons that may have been started during package installation within the chroot/directory. For example : grep /var/tmp/proc/mounts and then umount /var/tmp/proc and any others that may be listed. End by umount /var/tmp which should work without errors.

Detach the Disk

Return to your disk management interface in your Gandi hosting account, and detach the disk to which you have just copied, or on which you have just prepared the installation.

Make the disk bootable by associating a kernel

In the administration interface, you have the posibility to define a kernel for a disk. Select the virtual disk and associate it with a kernel suitable for the image you have just created. Let's take a "data" disk as an example, that we want to transform into a bootable virtual disk:

Use the link to change the disk information, and at the bottom of the page you will find an option to change the disk type:

You have now three advanced options, as for a bootable virtual disk:

By associating a kernel, the disk becomes usable as a custom image and will be shown in the list of images, just like the Gandi AI or expert images provided by Gandi during the creation of a new service via the administration interface.

Create a server from this image

During the creation of a server via your administration interface on the website, you will find your new custom image in the list of available images. The server creation will thus use this image as the source for the system disk. The server will then normally boot from a disk which is a copy of your image. As such, you have the possibility of creating serveral identical service from the same custom image.

Troubleshooting the server during boot

The Gandi hosting platform gives you the ability to access the command line of your server via the emergency console, which is accessible via an ssh session, and provides access to the console commandline of your server.

If you configure a getty on the console by default (tty1, xvc0, hvc0, depending on the chosen kernel version), you will have an emergency shell in case of boot errors, or a login prompt in the case of a successful server boot.

This emergency console will allow you to view the boot messages, and more importantly, any errors that may occur and thereby allowing you to debug your image.

Correcting the Image

Later, if you notice errors, or have forgotten anything in your custom image, you only need re-attach your new disk image to one of your servers (ultimately, and preferably, a server which was created from the same source image). Then you only need make the changes on the source image and then detach the disk. The image will thus be available to create new servers without error.

Gandi modification on standard OS

aegiap — Wed, 27 Oct 2010 18:03:00 +0200

What are the modification that Gandi staff makes on standard installed OS to be used on Gandi hosting?

Local modifications

Gandi hosting infrastructure is using Xen virtualization in paravirtualization mode (for the moment). As such we have to build a Linux kernel with specific options to allow it to boot your virtual server. Moreover as we allow customers to dynamically add or remove resources, the hotplug system in the kernel was patched by our team to allow a correct use of this features (mainly correct udev call). All kernel modules are available at each new kernel release on http://mirrors.gandi.net/kernel/. As of the 2.6.32, we now use upstream kernel source for building the xenU kernel and you should find the buildconfig file in /proc/config.gz on your virtual server. We add external patches such as drbd (before upstream integration).

Each Linux base system that we provide on Gandi hosting contains modifications by our team. For example, we removed services based on hardware clock as Xen does not provide direct access to it. On some distribution we had to disable boot features such as ureadhead or plymouth to allow a flawless boot of the virtual server. The main configuration is done during the boot process, especially the first boot process.

On a side note, on x86_32 architecture, to use the hardware capability of Xen, the libc could use the nosegneg hwcap with the correct libc-xen package.

Package gandi-hosting-vm

The idea which triggered this article was the release of a new version of gandi-hosting-vm. The package contains a collection of scripts to setup the local system of your virtual server at each boot and when specific events about hosting resources happen.

Changing hosting resources

When you add or remove resources dynamically to your virtual server, the Linux kernel receives information from Xen - the system managing all the virtualization. Each of these events are passed to the udevd daemon which apply configured rules to these events. It mainly creates files in directory /dev to allow access to the newly discovered resources.

For resources that Gandi hosting allows you to dynamically change, we wrote some udev rules (located in /etc/udev/rules.d/86-gandi.rules) to start a script when a virtual disk, a virtual interface or even a virtual cpu is attached (or removed) to the virtual server.

On a more detailed level, when a virtual interface is attached, the script /etc/gandi/manage_iface.sh is called by udev and a DHCP request is sent for this interface. A couple of other scripts setup the default route (/etc/gandi/dhcp-postconf) and store network configuration (/etc/dchp-hostname) in a tmpfs directory for further configuration at the end of the boot process. When the virtual interface is removed, the script simply removes the local network interface.

When a virtual disk is attached a similar script (/etc/gandi/manage_data_disk.py) is called. It tries to check the file system on the device or in its partitions (only in GandiAI mode) and mounts the file system in a specific mountpoint using the file system label /srv/<FS label>. If no label is setup on the file system, it uses the device or partition name as mount point (/srv/xvdc1 for example). To change the default mount options, please edit the variable mount_options in the beginning of the Python script.

During the boot process

The gandi-hosting-vm package provide two services called on boot : gandi-mount and gandi-config. The first one mounts already attached virtual disks in the local system in the /srv directory (see the description of /etc/gandi/manage_data_disk.py). You can start the service again once your server is booted and it will mount attached disks to the server (if you remove udev packages for example).

The second service starts a couple of specific plugins to setup your local system. Some of these configurations are optional and a configuration file is available for you to choose to setup each of these optional features /etc/default/gandi. Each configuration variable contains a short description in the default config file. These plugins are configuring the default local console for the hosting emergency console, configure the hostname and dns resolver, change the timezone to Europe/Paris, change the hwcap nosegneg according of your kernel version, change the motd to the default and so on.

For example, when the plugin 11-config_ssh is called, it creates SSH key for the local system if the keys are not already present. Then, depending on your configuration, it could add the Gandi SSH management key to the root user keyring (variable CONFIG_SSHMGMT) and reconfigure your sshd server by disabling password access for root, disabling empty password and enabling compression (variable CONFIG_SSHD).

Package gandi-hosting-agent

Gandi agent is used to setup the virtual machine according to customer information. In case of expert mode server, the setup of the local system is limited to setting the root password and creating the administrator user (as chosen by the customer) to avoid ssh-ing the server as root. In case of a GandiAI mode server, the agent uses specific modules to setup applications on the local system.

Once your expert server is setup after creation, you can remove gandi-hosting-agent packages. For example : dpkg -P $(dpkg -l | awk '/gandi-hosting-agent/ { print $2 }' | xargs) in deb based package system or rpm -e gandi-hosting-agent in rpm backed package system.

Hosting Public API 1.0 beta

Leland Vandervort — Wed, 20 Oct 2010 12:56:00 +0200

As you are probably already aware, we have been beavering away to offer you a public API to manage your resources on our Cloud hosting platform. In order to ease the work, several things have been reorganised so as to provide an interface which is easy to use, and allow a real management of your hosting resources at Gandi.

By way of this introduction, this article will be deliberately less technical, and will only succinctly present the elements that will be developed more in-depth when the official release of the API is launched.

API ?

For most human beings, the term API doesn't really mean a whole lot. Therefore, I suggest that we begin with a quick explanation of what an API is.

An Application Programming Interface (API) is an interface provided by a program or system. This interface allows other programs to use a standard machine to machine interface which in turn opens the possibility to create an appropriate human to machine interface. From the technical perspective, an API is a collection of functions, procedures, or classes exposed by a software library, operating system, or service. Knowledge of these APIs is paramount to the interoperability of various different software.

You can see the complete definition here.

Concretely, the Hosting API allows a user to manipulate and manage his various Gandi Hosting resources. For example, he may create new virtual machines, new disks, release IP addresses, etc.

Our API has been designed to be as simple as possible to use.

Use of the API

The API will allow you to manipulate all of your hosting "objects", from entire virtual machines, to components such as disks, network interfaces, IP addresses, etc., all in a logical and programmatic fashion. You will be able to create, delete, modify, and list the various objects.

One of the main objectives, however, is that it used by other software or programs in order to create something directly associated with us.

Let's look at two small examples:

One useful for everyone, thanks to the "vm.update" method, if you happen to detect in your log files that your server is in need of more memory, you could automatically add this memory to the virtual machine "on the fly".
An example for a company that produces software, thanks to "vm.create", you could potentially automatically deploy a server quickly from a disk that has previously been prepared with your application, and offer it directly to your customers.

The first set of methods will allow manipulation of the following objects:

datacenter - list , info

image - list, info

vm - list, info, count, create, update, start, stop, reboot, disk_attach, disk_detach, iface_attach, iface_detach

disk - list, info, count, create, create_from, update, delete

iface - list, info, count, create, delete

As well as management of operations:

operation - list, info, count, delete

New Features to Come

Taking advantage of the release of the API, several new features will also see the light of day. These are features for which you have been asking, for the most part, and which initially will be managed by our support team. Certain functions have been revised and move to areas where they would allow us to offer something new.

The choice of kernel and command-line options, for example, is now made on each disk. The disks, as a result, now have a more important role to play. It will also be possible to detach a former "system" disk, and to re-attach it to another virtual machine in order to, for example, recover the system following an unsuccessful upgrade. You could also choose from which disk a given virtual machine will boot.

How Do I Take Part?

The private beta phase has already begun. Simply send an email to beta-hosting AT gandi.net with your user ID (handle) and we will contact you with the details to allow access to the API.

Test Our Solution!

All European business customers at Gandi (and, initially, those subscribed to our newsletter) will find a coupon in their account allowing a free test-drive of our solution for a month. For everyone else, a request form for test shares is also available.

Be on the lookout though, as more information and updates will be made available in the coming weeks!

Kernel and cmdline

Ryan — Thu, 03 Jun 2010 11:00:00 +0200

We are pleased to announce that you can (finally!) choose the version of your kernel that you want (from a list which will be continually expanded), and associated boot options (cmdline). 2.6.18 and 2.6.27 are "base" versions supplied by Xen (backport of Xen patches for 2.6.27). Version 2.6.32, which is is currently available, uses paravirt_ops and the "Linux" implementation of Xen patches.

We will show you the new kernels here. They can be found in the "advanced mode" within the server's management page:

And you will then be able to access:

Concerning cmdline, you may now deactivate selinux at boot, boot as a single user, change the disk and the boot partition (which is practical for working with "images"), or choose the most appropriate console for your needs. In short, everything that you need to manage your updates in a more friendly environment, or to repair your server in the most autonomous manner.

If you feel that an option is missing, please let us know.

Mandriva 2010 image in alpha (updated)

aegiap — Fri, 21 May 2010 17:35:00 +0200

Server hosting by Gandi allow customers to choose from a selection of OS images available during the creation process of the virtual server. After the creation of the image by Gandi and internal testing, a new distribution is released to a specific group of hosting customers called 'alpha'. These clients can create server using these release candidate images. This allows Gandi to increase the types of testing and usage, and to find more bugs and problems by working with a small group of its customers.

Today - May the 21th - the Mandriva 2010.0 image has been released . This new version of the Mandriva distribution boots with a 2.6.27 kernel by default. It is currently only available for the 'alpha' customer group but will shortly be available for all customers. Please contact us if you wish to participate in our alpha testing phase.

16th August 2010 : Image is now available for everybody

Cherokee Arrives at Gandi

Leland Vandervort — Thu, 29 Apr 2010 13:48:00 +0200

(Translator's note: Taskigi Sequoyah would be proud! ;) )

A friend had explained to me that he had just finished installing a web server. Having decided on a change and not to use the mammoth Apache that everyone knows and loves, my friend seemed to have found a decent alternative. Bearing the name Cherokee, it appeared much leaner and with greater performance than Apache. See the benchmarks from the project website at the end of this article.

Personally, I had no knowledge of Cherokee, though I had certainly heard the name pandered around once or twice, but I never really paid it any attention at the time. So to give it a whirl, I decided to install it locally, which to my surprise exceeded my expectations. On a debian (or derived) distribution, installation is achieved by a simple apt-get install cherokee (or 'aptitude'...)

The web management interface provided is clean and intuitive. The basic idea is quite interesting; It is possible to activate the PHP interpreter with a simple mouse click; same for activating different virtual hosts as well as other options. There are also wizards to assist with the installation of such applications and frameworks as Django, Rails or Wordpress, etc.

I thought that it would be interesting to present this project during a developers meeting at Gandi, especially for those who use GandiAI. The presentation was well received. After a few pow-wow, and a glance through the source code, there was nothing really unusual or risky. This doesn't mean, of course, that there aren't any bugs in it, but at any rate clean code is a good sign! The core is written in C with the administrator interface written in Python.

Thus is the way in which Cherokee made its appearance at Gandi, and a server image was produced. In this way you can rapidly test a server with Cherokee pre-installed.

We have also added an "expert" mode distribution with Cherokee pre-installed... You need only to select this distribution during the steps to create your server. And, just as a reminder, in case you haven't yet tried our service, you can always request a free trial using our online form!

For further information about Cherokee, you can visit the project website. Additionally, you can find a comprehensive article about Cherokee in the Gnu/Linux magazine France, written [in French] by Carl Chenet in issue number 125.

Gandi 10th Anniversary - The Experience

Leland Vandervort — Wed, 17 Mar 2010 19:33:00 +0100

To celebrate Gandi's 10th anniversary, this hair-brained idea to give away, in ten days, 55000 domains, raise a very practical question. How, once we open the floodgates on such an operation, to maintain the highest quality of service on the site? The festive spirit could well have transformed into a nightmare for our customers if they were suddenly unable to access their management interface.

So we took the decision to host the event on a dedicated site. This was a hitherto dreamed of occasion to put ourselves into our customers' shoes, and use our hosting infrastructure for this event. We defined the rules of play: Using only the tools provided to our customers, build an architecture which was easily scalable and didn't break the bank, and to demonstrate our renowned flexibility.

Keep it Simple, Stupid.

We had the "luxury" of one week from design to implementation. As a result, the charming idea to demonstrate our "cloudlike" site based on modern technology was summarily put out of our minds. To be perfectly honest, given the time scale, the lucky chosen developer had a nifty precept to select the technology: "You have full choice of the technology, but you have one week." It would be PHP/MySQL, which isn't exactly everybody's favourite! It would, nevertheless, allow us to release a tested site within the tight time frame.

To adequately sustain the load generated by such an event, several servers would be needed. We then hit our first stumbling block: Gandi does not [yet] have a load balancing solution for the hosting solution! Nevermind, we'll use the old yet faithful round-robin DNS method, with a low TTL to be able to quickly remove a front-end server from production in case of an incident.

Our Linux distribution for this occasion would be Ubuntu 9.10 - because it is reasonably up-to-date, with the 2.6.27 kernel.

Small Servers - Go Forth and Multiply!

The best way to sustain the high loads for our solution is to split the functionality among several small servers. This way we would maintain a minimum level of "vertical" scalability (you can dynamically increase the memory and CPU allocated to a server), and the architecture provides "horizontal" scalability.

In this way we could easily add resources if we started to feel the pinch, and add or migrate shares from one server to another as load requirements necessitate. There are numerous advantages of using multiple small servers:

each server gets a minimum of one core, burstable, of the CPU (yes, one whole core, even with one share -- that's new, by the way!)
assured resilience, with shares spread somewhat randomly across a few hundred different physical servers.
specifically in a virtualised environment, the memory performance is best with less than 1GB of memory.
if you have 4 servers of one share each, rather than one big server of 4 shares, they can dynamically increase to 8x4 shares, or 24x4 shares with a reboot, and all of this without modification to the architecture. A big server, however, would only scale to 8 shares without a reboot, or 24 after reboot.
resources may be easily moved towards servers that need it most

We commence with a simple architecture:

24 (!) servers of one share each, to manage the PHP website: 10 for the English site, 10 for the French site, and an additional 2 of each for IPv6.
2 servers of 4 shares each for replicated memcached to reduce database load and manage sessions.
1 MySQL server of 4 shares, which contains the pre-generated promotion codes (they actually all fit within memory, so the database itself should really be pretty bored doing nothing...)

After a couple of lovely overloads and a bit of code review, the database would finally be greatly spared by memcached (see the section "lightweight coding...").

One of our administrators would put his fingers to work on the site to create and configure 24 servers -- at the same time! Obviously the release of the hosting API or an admin interface function would have been welcome. (Thanks to cssh in this case!)

Lock Down (somewhat) the Machines

A default installation always needs a few finishing touches. The very fact of opening a MySQL database on the "public" network made us a little edgy. So, swooping 'netstat' and shutting down non-critical services listening on public ports. With the help of tcp wrappers (hosts.allow, hosts.deny), all of the "private" interfaces are also locked down (sshd, mysql accessibly only from the web farm).

Finally it behooved us to pay close attention to the PHP code and MySQL queries; The safest way to avoid php code injections is to bind all the parameters after a prepare(). This also helps reduce load on the database when several execute() are called.

One important detail: since the site should allow a user to send an email to any "arbitrary" address, it was absolutely critical to limit its potential for abuse by some clever black-hat as much as possible. At the very minimum, the number of sent emails per promotion code was limited, in addition to very close monitoring.

Setup the Development and Deployment Environment

The sharing of data between the sites in effect adds a single point of failure, as well as a potential architectural bottleneck. As such, we decided to deploy the content of the site locally on each of the servers. We would use one server for developing and staging, and ultimately for the development and testing of updates. A quick script and some 'rsync' would allow rapid deployment across the entire front-end architecture. Simple! (some would say ;) )

Resource Monitoring

A few moments before the operation, more as a precaution rather than a cure, all of the virtual machines from one to two shares. Using the statistics interface, from day one, one can see that the the virtual machines were essentially sitting "twiddling their thumbs" from boredom ;) :

It would have been cool, at this very moment, to reduce back to a single share per server, or make use of Gandi "Autoflex", or even given the actual load observed, set up scheduled flex for each hour to hand out the promotion codes! Unfortunately, with all hands on deck, we missed this opportunity to demonstrate this [econono-techno-ecological ;)] feature.

Lightweight Code is Worth More than a Thousand Beefy CPUs

Even though we physically had several thousand CPUs and a few Terabytes of RAM at our fingertips, Tuesday turned out to be somewhat chaotic and worthy of note here. After Monday, which managed the load very well, the "smooth" execution of our one and only SELECT COUNT brutally altered and became excruciatingly slow (300ms). We had naively thought that this "only" query, on a table held exclusively in memory, wouldn't be an issue. As such, it was executed on every page of the site. The multiple simultaneous accesses to the database, coupled with the UPDATE operations for the promotion codes, resulted in the database, despite the near-idle system performance, started causing database lock contention.

The usual knee-jerk reaction to such a situation is to increase the number of shares to support the load. It's great for a quick-fix temporary solution, but it's not enough!

A new analysis of the system, questions about the code, and the use (or salvation) of memcached resulted in recovering the optimal performance. Equally, a modification of the database queries used probably would have been prudent.

The moral of the story: the code, indexes, architecture (etc.) are the cornerstones of your ability to support usage load, and if they are "CPU friendly", they will save the day. Otherwise a catastrophe could be lurking, or at the very least, the unnecessary purchase of additional shares.

Also, as we said earlier somewhat tongue-in-cheek -- it's eco-friendly!

Some Numbers

36 shares total, but we could have done it with less (*sniff*)
5% CPU usage at peak
4000 requests per front-end web server in the first minute of each hour (roughly 1400 requests/second total)
a minimum of 11 seconds to hand out 1000 promotion codes.
a maximum of 40 minutes to hand out the same number of promotion codes, during the Tuesday incident described above.

What to do if your server stops responding?

Ryan — Wed, 13 Jan 2010 13:26:00 +0100

As you probably already know, our platform protects you from hardware failures that might occur on your server.

In the event of a problem on the machine, or if we suspect that a problem might occur (abnormal temperature, corrupted memory, etc.), your “server” will automatically be migrated to another machine. However, if you have an internal problem on your "server" that is not to due to the physical machine, and if it no longer responds, then you will need to take action.
The first action to take is to be sure that your server’s status is shown as “Running” on your Gandi interface. This is because the status may also be “Stopped” or “Paused” if, for example, it has not been renewed. So given that it is "Running" and non-responding, here's what to do.

There are three different cases:

1. You can still connect to your server via SSH. In this case, the following commands will help you analyze the situation:

"uptime" will give you the current load of the machine,
"free", will show you the amount of memory used by your applications in the “used” column,
"top" (we recommend that you install “htop") will show you the ranking of applications in realtime ordered by their use of resources (memory, CPU),
"dmesg" shows you messages from your linux server’s kernel,
Consulting logs such as /var/log/messages or /var/log/daemons with the command, “tail”, for example (tail /var/log/daemons) will also provide you with precious information.
"df –h" shows you the amount of disk space available on your disks.

The most frequent causes of error are:

No space left on your system disk: this situation is often caused by a lack of appropriate log management on the server, or by a database which fills up too quickly. The solution is to clean up wasted space, or enlarge the disk (see the guide on this).
Not enough RAM on the server, or too much memory used: the simple solution is to add more RAM by adding additional shares. If you have an expert server, you can also try to modify the behavior of the Linux’s available memory by using the command 'sysctl -w vm.overcommit_memory = 2'. Warning: so that the modification can be maintained following a reboot, you must also add "vm.overcommit_memory = 2" to the "/etc/sysctl.conf" and "/etc/gandi/sysctl.conf" files.
Too may processes are running simultaneously on your machine: you will need to lower the values in the configuration files of your applications (the number of simultaneous connections on Apache, for example) or increasing the power of your server by adding shares.

2. You can no longer connect to your server under SSH, and it does not respond to ping or it has a slow response time.

The virtual console which you may activate from your account (guide available), gives you direct access to your machine as if a monitor and a keyboard (still virtual) were directly attached to the server.

In this case, you can stop all the applications that are causing problems, and once again get access to the server.

The 'sysreq' shortcuts are available from the console of your server. The commands can be given by pressing Ctrl and “o” (as Oscar) in order to enter the sysreq mode, and then you can enter the command. By doing this, you can stop all the processes by: Ctrl+o +i (to kill). Ctrl+o +h gives you quick help to all of the available sysreq commands.

3. Your server may be “unreachable” but may nonetheless be working normally, and without any technical problem.

This may happen if your server is the victim of a DDoS attack for example. Your server will therefore be isolated from the rest of the network in order to protect our infrastructure and the quality of service for other customers.

You may verify whether or not your server is in this state by performing a “traceroute” command on the IP address of your server. If it stops at its arrival at Gandi, on one of our routers for example, it is likely that your server has been isolated from the network. You may then connect to your server via the console, though you will need to contact the support team to correct the problem (they will usually contact you first). This sort of isolation happens only very rarely.

After that, if you are still blocked, then this means that you are in the 4th case: your very own case ;-). Please send an email to our support team indicating that your server has been blocked, and you will get an answer as soon as possible.

How to turn your website into a "Web Infrastructure"

Ryan — Wed, 13 Jan 2010 13:25:00 +0100

Many websites start with a single server solution, a box acting as a web server and database server all in one. Simply, easy, cheap. The problem comes when traffic gets too high (a victim of their own success!). Many customers want a bigger box, but the answer is actually changing your architecture from "web server" to "web infrastructure". You can duplicate web servers, use the DNS to load balance them and ramp up your capacity very fast and very far.

For simplicity's sake, let's take a real example. As some of you already know, we support the Millenium association in their promotion of online video games.

The increasing success of the website, millenium.org, made us re-design the architecture of the website so that it could handle the numerous videos shown on the website to its 17,000 unique visitors per day far more easily and efficiently.

Following a major update of one of the games served by Millenium (WOW patch 3.1), we increased the server power to 16 shares in anticipation of an increase in load. Our expectations were quickly exceeded with over 50,000 unique visitors the first day, and just as many over the following days.

The website received between 500 and 1,000 simultaneous visitors and a large number of videos, which is not viable for a single LAMP server. We immediately changed the infrastructure, by moving from a unique-server model (which is often the starting choice) to an infrastructure-based model. We went from a vertical system (more power) to a horizontal system (more servers):

As you can see in the diagram, we moved the database to a separate server and duplicated the web server to two machines. The load was therefore split by the domain's DNS (using a simple DNS round robin technique). We could also move to dedicating a 1-share server to load distribution, though such a solution would take a bit longer to implement. In our case it took 2 minutes to add the shares to the account, 6 minutes to create the 2 servers, 10 minutes to transfer the data, and 2 hours to configure the services.

Today, the platform easily handles 1 million unique visitors per month, for over 3 million pages viewed - a good thing! Best of all the platform is now capable of evolving. If the database suffers, all we need to do is to add more shares to the database server. If the web front starts to become saturated, we just need to add another.

Because our cloud infrastructure allows you to create as many servers as you want from the hosting resources in your account, you can always add more, and always change your web architecture without replacing physical servers. The VPS hosting system is flexible and allows you to increase from a 1-share server (1/64 of a machine + 1/64 as reserve, 256MB of Ram) to a 16-share server (1/4 of a machine + 1/4 as a reserve, 4GB de Ram) at any time and as often as you want.

If you find yourself confronted by this type of problem, please feel free to contact us, as we would be more than happy to help you.

Gandi Hosting Team