We all knew that we are risking with MMM. Risking, and placing availability as a more important like consistency. But non of us can risk loosing data forever but we show using it, regarding to our conversations think: "I can fix my data later on, but I can’t turn back time and prevent the downtime. (Pascal Hofmann@xaprb.com)".
As I wrote before about staying online, now let me write about how to stay consistent.
We all know, mmm is not like a key of salvation, but its getting close to it . While MySQL doesn't support multi-master-slave environments from it's source code, we will sleep badly wondering on the safety of our precious databases.
But its not just about MMM, a few days ago we ran in to a well known InnoDB "feature". Its about the auto increment counter determination on restart. InnoDB try to count the next auto increment value on MySQL restart what can screw up things in the replication as in your data integrity too (http://dev.mysql.com/doc/refman/5.1/en/innodb-auto-increment-handling.html) what could be risky when you are about to use mmm, because you can restart your masters whenever you just want to.
In this post I'm using maatkit to verify and restore the rows without locking or downtime.
In this blog, I would like to mark myself as a power-user of MMM. For more then 3 years now I'm engineering high-traffic websites and none of them was a small project. In the beginning I worked behind the livejasmin.com project (warning: NSFW!) which is an Alexa top100 site and now I'm working at ustream.tv which is in the top500 (so far). In this area, your greatest enemy is downtime. Thousands of users want to see your dynamic content by executing thousand of queries against your MySQL servers and what they don't have is the patience and they don't even tolerate critical security-upgrades. In both sites I mentioned above there is no point in time during the day at which less then 10k visitors are hitting the sites. Now one question comes up: How can you do an upgrade on your MySQL servers or how can you alter a larger table?
So you have a new feature and that means another index on a table or you just want to upgrade your application, there's no chance that your downtime will be hidden. We all know, even if you run your site with no downtime for months, 1 minute can make your bosses forget about that and ask you THE question: "What has happened?". If you would say: "There was no problem, I just took down the site to upgrade MySQL", you won't be the employee of the month.
So let's see how can MMM help:
This is the first post on the brand spanking new MMM for MySQL Community Blog. Expect posts on new features for this great project. Expect user experiences.
So, your question is of course: what is MMM for MySQL and why haven't I heard of this before?
Well, MMM for MySQL is a project that strives to provide a HA Solution for MySQL, with automatic failover between multiple masters and slaves. If set up properly, you don't have to worry anymore over downtime for upgrading to a new version of MySQL, or a new version of your application (=> large alter tables, addition of indexes, etc.)
The project has recently reached it's version 2, which is a complete rewrite from scratch. We are now slowly seeing this being put into production, with some success stories making us happy already.
I won't make it too long here, but look forward to hear more from us!