I like to hear from organizations using products not just the suppliers of those products. I believe that gives a more complete view of a product than a simple conversation with the supplier could offer. David Blinder, CTO of Familybuilder, let me know a bit about his experiences with codeFutures' dbShards (see codeFutures dbShards for more information about the supplier.) Thanks for taking the time to communicate with me, David.
Please introduce yourself and your organizationMy name is David Blinder and I am the CTO of Familybuilder. Our company develops family-oriented social network applications. Our flagship application is Family Tree on Facebook, boasting over 35 million users with over 5 million monthly active uniques. The application is very user centric and data intensive, housing information on family relations, both in network and out. Family Tree has been the top application for families in the social space for several years. Our company is considered a seasoned start-up, at this point, with ongoing and far reaching potential.
Family Tree started out as is typically the case - in a conventional hosting environment. Initial issues with a code base strewn with inefficiencies plagued our growth but were remedied and re-factored early in our history. The insult came when we hit a wall with our conventional hosting choice and its restrictive downsides. Calls to NOCs at all hours to get updates on machine builds as well as restarts and a world of other communication issues led us to investigate the growth in cloud computing. We ported our applications over to Amazon's cloud and leveraged EC2 and S3 immediately to support our rapid growth.
What were you doing that needed this type of technology?Our application was a great fit in the social network space. Users seeking to create subsets of friends into more meaningful classes of relationships found our application appealing for family interaction/communication and suggested it to other relatives.
I recall an ad that we placed within Facebook years ago that brought 30,000 users one day and crippled our systems. The following day, while we attempted to recover, we received over 45,000 users virally.
Our need grew, ultimately, from stress to a traditional MySQL Master/Slave configuration that we outgrew on Amazon, both in the robustness of their instances and in the I/O on their extra large instances. I like to call this the “cloud I/O barrier.”
Simply put, we reached the end of the line. It should be noted that this was before the release of Amazon’s RDS product and the XXLarge instance class they offer now. Even if RDS was available, we do not believe that technology would perform anywhere near as well as dbShards for our application.
Knowing about how EC2 instances are provisioned is vital to success on the cloud. There are twists and we looked at all options before we made our choice to move to a solution like CodeFutures dbShards.
What products did you consider?We looked at Cassandra, Oracle, MySQL Cluster but none were as straight forward in design and as cost effective as the CodeFutures product.
Why did you select this one?Most importantly, dbShards gave us the scalability we needed, transparently without a lot of application changes. The Relational Sharding approach is ground breaking, allowing us to preserve and work with our data in a fully relational way. The product was also provided, on a very cost-effective basis. Licensing an Oracle product is start-up suicide. There is a place for it but not here and not now.
We needed a solution that would get us up and running fast as well as jive with our development wing. Consider that we were already using MySQL and much of our reusable code base was written around it already. We liked the CodeFutures product because the connection to MySQL through PHP was transparent — their plug-compatible Driver works just like the standard database vendor driver. There was some refactor required but, for the most part, our team moved with ease to create a new release that is scalable 20x or more.
Another key benefit in the choice, which was realized fairly immediately, was their support and dedication to our project. We were an early adopter and there was risk but it was calculated. Their support through our code re-factor was excellent all the way through to a cutover lasting 12 hours.
What tangible benefits have you received through the use of this product?The release of our application with the sharded data model has enabled us to grow our application user base far beyond prior levels. We often look at the numbers and recall the limitations we faced. People can use our application and enjoy the experience and we know we can shard out further if we need to - it's a good feeling. And our marketing team can promote as much as they like, everything is solid and handles the load well. We are handling our load with 4 shards today (8 servers with complete redundancy and failover), and can easily scale to many times this level when needed.
What advice would you offer others facing similar issues?I would say to look as deeply into your source and data as you can and optimize as far as you can. I would recommend you take advantage of Memcache. After you have gone as far as you can go with the code, then move to a full sharding solution. Doing everything I recommend prior to sharding your data will prepare you for the move.
I would say that PHP application developers stand to gain from the use of the CodeFutures product as it was easy to integrate and virtually transparent.
What are you looking forward to in the future releases of the product?
- Cross shard transactions
- Shard hints (this has been around for a while, but we are just implementing it)
- Config-directed Cache support
codeFutures has published a more detailed profile. It can be found here: