The Piwigo.com Blog

Piwigo.com subscription form gets a full redesign, with 3 goals in mind: improve VAT management, give choice between Individual and Enterprise plans, make it possible to subscribe for several years.

1) VAT and european laws

Piwigo.com hosting service is managed by a French company. Clients are coming from all over the world, among more than 70 countries. Depending on your country, the VAT (Value Added Tax) does not apply in the same way. Furthermore, if you or your organisation has a VAT number, other rules apply. No need to go further into technical details. Keep in mind that we need to collect a few data to accurately give VAT back to appropriate countries.

Piwigo.com subscription: we need a few data from you!

As long as we don’t have them, the subscription will keep asking for a few data.

Piwigo.com subscription: give your country and your VAT number, if you have one.

Side note: this new rule dates back from 2016 and we were not always asking for your country/VAT number. Until now, we guessed your country based on your IP address and we considered Enterprise clients had a VAT number and Individual clients had none. From now on we won’t have to guess anymore!

2) Choice between Individual and Enterprise plans

Piwigo.com subscription: are you an individual or an organisation?

In its early days Piwigo.com was only selling a 39 euros (per year) plan, for individuals. Since 2017, Piwigo.com Enterprise plans, now official! Enterprise clients still had to contact us to create an order. The new subscription form gives organisations the power to create Enterprise orders by themselves, without needing help from Piwigo.com support.

Piwigo.com subscriptions: several options for the Enterprise plan

3) Directly subscribe for 2 or 3 years

Piwigo.com subscription for Individual plan: select your duration

Interesting for Piwigo.com: it will obviously give us the opportunity to increase our available cash. More cash in the bank makes some investments possible, like recruitment.

Interesting for clients: by taking 2 years, you get a 10% discount, or 8€ less on your bill. By taking 3 years, discount is 20%, meaning 23€ kept in your pocket. As it is very usual our clients stay 5 years or more, we think it is a good deal!

The huge downtime at OVH that occurred on November 9th 2017 was quite like an earthquake for the European web. Of course Piwigo.com was impacted. But before that, we lived the server failure of October 7th and another one on October 14th. Let’s describe and explain what happened.

Photo by Johannes Plenio on Unsplash

A) October 7th, the first server failure

On October 7th 2017, during saturday evening, our “reverse-proxy” server, the one through which all web traffic goes, crashed. OVH, our technical host, has identified a problem on the motherboard and replaced it. Web traffic was routed to the spare server during the short downtime. A server failure without real gravity, without loss of data, but which announced the start of a painful series of technical problems.

B) October 14th, a more serious server failure

A week later, on October 14th, the very same “reverse-proxy” server saw his load go into such high levels it was unable to deliver web pages… Web traffic is again switched to the spare server, in read-only mode for accounts hosted on this server. About 10 hours of investigation later, we were still not able to understand the origin of the problem. We have to decide to switch the spare server to write mode. This decision was difficult to take because it meant losing data produced between the last backup (1am) and the switch to spare server (about 8am). In other words, for the accounts hosted on this server, the photos added during the night simply “disappeared” from their Piwigo.

This is the first time in the history of Piwigo.com that we switch a spare server to write mode. Unfortunately, another problem has happened, related to the first one. To explain this problem, it is necessary to understand how Piwigo.com servers infrastructure works.

That’s enough for infrastructure details… let’s go back to October 14th: so we switched the IPFO to use the spare reverse-proxy server. Unfortunately, we met 2 problems in cascade:

1. the spare reverse-proxy server, for one of the server pairs, pointed to the spare server
2. this very spare server was configured in write mode instead of read-only

Why such an unexpected configuration?

Because we sometimes use the spare infrastructure to do real-life tests. In this case, these were IPV6 tests.

What impact for users?

During the many hours when the web traffic went through the spare reverse-proxy server, accounts hosted on the faulty server returned to the state of the previous night where photos added during night & morning had apparently disappeared but they were able to keep adding photos. This state did not trigger any specific alert : the situation seemed “normal” for the users concerned and for our monitor system. When the problem was detected, we changed the reverse proxy configuration to point back to the main server. Consequence: all the photos added during the downtime apparently disappeared.

What actions have been taken after October 14th?

1) Checks on reverse-proxy configuration

A new script was pushed on production. It checks very often that reverse-proxy is configured to send web traffic on main servers only.

2) Checks on write Vs read-only mode

Another script was pushed to production. This one checks main servers are configured in write mode and spare severs are in read-only mode.

3) Isolate third-party web applications

The “non-vital” web applications, on which we have less expertise, were switched to a third-party server dedicated to this use: 2 WordPress blogs, wiki, forum and piwik (analytics for visits). Indeed, one of the possibilities for the server failure, is that an application entered the 4th dimension or was under attack. Moving these applications into an “isolated” server helps to limit the impact of any future issue.

4) New backup system

The decision to switch a spare server to write mode, ie turn it into a main server, is a hard to take. Indeed it means giving up any hope to return to the main server. This decision is difficult because it involves accepting a loss of data.

To make this decision simpler, two measures have been taken: first to define a time threshold after which we apply the switch. In our case, if the failure lasts more than 2 hours, we will switch. Then backups must be more frequent than once a day: if the backups were only 1 or 2 hours old, the decision would have been much easier!

In addition to the daily backup, we have added a new “rolling backups” system: every 15 minutes, the script analyzes each Piwigo on specific criteria (new/modified/deleted photos/users/albums/groups…). If anything has changed since the last backup, the script backs up the Piwigo (files + database) with a synchronization on the spare server.

C) What about the giant downtime on OVH network, on October 9th and 10th ?

Being hosted at OVH, especially in the datacenter of Strasbourg (France, Europe), the downtime has greatly impacted our own infrastructure. First because our main reverse-proxy server is in Strasbourg. The datacenter failure put Piwigo.com completely out of order during the morning of November 9th (Central Europe time). Then because we could not switch the IP Fail Over. Or rather, OVH allowed us to do it, but instead of requiring ~60 seconds, it took ~10 hours! Hours when the accounts hosted on the reverse-proxy server were in read-only.

Unlike the October 14th situation, we could not make the decision to switch the spare server in write mode because an IPFO switch request was in progress, and we had no idea how long it would take OVH to apply the action.

The Piwigo.com infrastructure has returned to its normal state on November 10th at 14:46, Paris time (France).

OVH has just provided compensation for these failures. We were waiting for it to publish this blog post. The compensation is not much, compared to the actual damage, but we will fully transfer this compensation to our customers. After very high level calculations, 3 days of time credits were added to each account. It’s a small commercial gesture but we think we have to reverse it to you as a symbol!

We are sorry for these inconveniences. As you read in this blog post, we’ve improved our methods to mitigate risk in the future and reduce the impact of an irreversible server failure.

17 days after Piwigo 2.9.0 was released and 4 days after we started to update Piwigo.com, all accounts are now up-to-date.

Piwigo 2.9 and new design on administration pages

As you will learn from the release notes, your history will now be automatically purged to keep “only” the last 1 million lines. Yes, some of you, 176 to be exact, have more than 1 million lines, with a record set to 27 millions lines!

A. The short version

On April 27th 2017, we replaced one of Piwigo.com main servers. The replacement itself was successful. No downtime. The read-only mode has lasted only 7 minutes, from 6:00 to 6:07 UTC.

While sending the notification email to our clients, we encountered difficulties with Gmail users. Solving this Gmail issue made the website unavailable for a few users and maybe an hour. Everything was back to normal in a few hours. Of course, no data has been lost during this operation.

The new server and Piwigo are now good friends. They both look forward to receive version 2.9 in the next days 😉

B. Additional technical details

The notification message had already been sent to the first 390 users when we realized emails sent to Gmail addresses were returned in error. Indeed Gmail now asks for a “reverse DNS IPv6”. Sorry for this very technical detail. We already had it on the old server so we added it on the new server. And then start the problems… Unfortunately the new server does not manage IPv6 the same way. A few users, on IPv6, told us they only see “Apache2 Debian Default Page” instead of their Piwigo. Here is the timeline:

• 6:00 the upgrade starts, switch to read-only mode
• 6:07 web traffic redirected to the new server
• 7:40 the new server is doing fine, we start to notify users
• 7:52 Gmail errors, we stop the notification
• 8:31 we add an IPv6 on the new server
• 10:05 following a few users feedback, we remove the IPv6

Unfortunately adding or removing an IPv6 is not an immediate action. It relies on the “DNS propagation” which may take a few hours, depending on each user.

We took the rest of the day to figure out how to make Gmail accept our emails and web visitors see your Piwigo. Instead of “piwigo.com”, we now use a sub-domain of “pigolabs.com” (Pigolabs is the company running Piwigo.com service) with an IPv6 : no impact on web traffic.

We also have a technical solution to handle IPv6 for web traffic. We have decided not to use it because IPv6 lacks an important feature, the FailOver. This feature, only available on IPv4, let us redirect web traffic from one server to another in a few seconds without worrying about DNS propagation. We use it when a server fails and web traffic goes to a spare server.

In the end, the move did not go so well and we sweat quite a this friday, but everything came back to normal and the “Apache2 Debian Default Page” issue eventually affected only a few people!

In the shadow of the standard plan for several years and yet already adopted by more than 50 organizations, it is time to officially introduce the Piwigo.com Enterprise plans. They were designed for organizations, private or public, looking for a simple, affordable and yet complete tool to manage their collection of photos.

The main idea behind Piwigo.com Enterprise is to democratize photo library management for organizations of all kind and size. We are not targeting fortune 500, although some of them are already clients, but fortune 5,000,000 companies!

Piwigo.com Enterprise plans can replace, at a reasonable cost, inadequate solutions relying on intranet shared folders, where photos are sometimes duplicated, deleted by mistake, without the appropriate permission system.

Introduction to Piwigo.com Enterprise plans

Why announcing officially these plans today? Because the current trend obviously shows us that our Enterprise plans find its market. Although semi-official, Enterprise plans represented nearly 40% of our revenue in February 2017! It is time to put these plans under the spotlights.

In practice, here is what changes with the Piwigo.com Enterprise plans:

1. they can be used by organizations, as opposed to the standard plan
2. additional features, such as support for non-photo files (PDF, videos …)
3. higher level of service (priority support, customization, presentation session)

Discover Piwigo.com Entreprise

We’re starting a new project: an “always fresh” Piwigo demo, with photos coming from Piwigo users, all over the world. Read all details on the announcement on Piwigo.org.

On the demo, you will see the contributor and a link to his Piwigo

Contribute to Piwigo plugin was installed on Piwigo.com, you can activate and use it. For now we’re talking about the demo on Piwigo.org. Later we will update the Contribute to Demo plugin to let you select the demo you want to contribute to, including Piwigo.com demos.

We hope you’ll love the idea and decide to become a contributor yourself!