In late May the Bazaarvoice team was delighted to speak again as a part of Database Month in New York City, and we were excited to speak about our recent work with Cassandra & Elastic Search. We discussed our goals for replacing the Bazaarvoice data infrastructure, as well as our hopes for the new system, and then we dove into the internal details of how we’re using Cassandra and Elastic Search to handle the scale needed by the myriad Bazaarvoice applications. We had a lot of fun at the talk as well as answering questions for quite some time aftewards, and we’re always excited to talk about this even more.
Author Archives: RC Johnson
Scaling on Mysql: Sometimes you’ve gotta break a few eggs
We recently delivered this presentation titled “How to Scale Big on MySQL? Break a Few Rules!” as part of Database Week here in New York City. The presentation is a lighthearted, and informative take on how Bazaarvoice Engineering has been able to take MySQL to billions of requests per month. The slides and video are available over at LeadIt.us. In the presentation we cover denormalization, query planning, partitioning, MySQL replication, InfoBright’s take on a data storage, and thinking beyond the RDBMS. Overall the presentation is a little over an hour, and is littered with great questions. I had a great time delivering the presentation, and I got a lot of very good feedback so I hope it proves useful for you as well.
Engineers Giving Back at Random Hacks of Kindness
On December 3rd and 4th Bazaarvoice was the lead sponsor on an event in Austin called Random Hacks of Kindness (RHoK), a coordinated worldwide hackathon for social good. The event started Friday night with a reception for all of the hackers at the Volstead Lounge where over over 60 people celebrated, heard a few quick thoughts on how technology could solve some big problems, and of course had a drink. Representatives of the Chicago Community Emergency Response Team, Williamson County Office of Emergency Management, NASA (a global sponsor for RHoK) and others gave a quick preview of the projects they hoped would be completed during the hackathon, and our own Scott Bonneau (VP of Engineering) spoke about the power that engineers have to change the world. Scott reminded us all how only a few short years ago, creating a new technology meant years of R&D efforts by large teams of highly educated scientists, and that now, anyone with a laptop and a credit card can launch an application over the course of days.
Saturday morning kicked off with coffee and presentations by each subject matter expert on the problems they had been researching to the over 50 hackers who were eager to get started. Additionally, NASA had a special surprise for the Austin attendees, and had arranged for Astronaut Ron Garan to speak about how his time on the international space station had provided him with a unique view into the power RHoK hackers have and the need for greater collaboration on the biggest problems of the world. The hackers organized themselves into teams based on their skillsets and desires, and before lunch the product design had begun. Teams were well fed throughout the weekend with great local food from P. Terry’s, The Peached Tortilla, and Freebirds, ensuring that no one ever went hungry or was lacking caffeine. Several teams coded late into the evening at the Capital City Event’s space, lounging on couches, and some even stayed the night.
By the end of the hackathon Sunday afternoon, the teams had built a number of amazing applications, and you can read more about the applications on the RHoK web site. All of the teams presented their work, and the top teams as selected by our judges received some amazing prizes (iPads, Kindles, and Buckyballs). Overall we’re very proud to have helped support this amazing opportunity, and we couldn’t have done it without the generous help of our sponsors and partners Homeaway.com, Freebirds, Capital City Events Center, and Github, as well as the numerous volunteers from tech companies throughout the community.
Not that our event went perfectly, but we certainly learned a few lessons along the way:
- Lead the leaders. You cannot run something of this size on your own. You need a team of leaders who’ll run alongside you and carry the ball for you in specific areas.
- It takes an army. You can never have enough volunteers. Find them early and have roles clearly lined out.
- It’s all about the network. It’s not about who you know, it’s about who they know. Find the connectors in your target demographic and pursue them. They’ll connect you with the masses.
To find out more about the next RHoK Austin, just follow the Pixadillo @rhokaustin. Want to help run RHoK Austin in the future? Send us a message @rhokaustin and we’ll connect you with the steering team for the next RHok Austin.
RHoK’ing Hackathon coming to Austin on December 2-4
In June, I travelled to Seattle to participate in Random Hacks of Kindness #3 (RHoK for short, which is typically pronounced “rock”). RHoK is a hackathon like many others, a chance to develop on a project for 24-36 hours and show off your application to the other hackers, but RHoK applies a slight twist to the traditional hackathon. Instead of trying to launch a new company in the span of three days or being confined to one particular API, RHoK aims to produce new ideas and solutions for social good, specfically producing software (and hardware) that will help those in crisis situations or dealing with climate change. RHoK is a 2-year-old event, holding worldwide coordinated hackathons at over 20 sites over the same weekend. Historically locations like Toronto, Bangladesh, New York City, and many others have hosted RHoK. RHoK Seattle was graciously hosted by Microsoft who allowed us to use several large conference rooms on their Redmond campus throughout the course of the weekend.
At the kickoff reception representatives of NASA, CrisisCommons, Microsoft and others talked about the need for tools to help first responders communicate effectively and applications to support those faced with a crisis (for example floods or earthquakes). Additionally, team members from NASA talked about the Open Government initiatives, which have produced large amounts of freely available data as well as the desire for NASA to produce more open source projects from our government. The reception was a great chance to meet other hackers from the Seattle area, along with a number of people who came in from Portland and California as we mingled and listened to music by DJ Maxx Destruct.
Starting with coffee and bagels on Saturday morning, we went through an exercise to determine the available skills of everyone at RHoK Seattle and then moved to start identifying the needs of the various problem definitions provided by the RHoK core team. There were representatives available for several of the problem definitions and the room gradually coalesced into a half dozen teams, each focused on a different problem and solution. The problems attacked at RHoK Seattle included the creation of mobile sensors that can easily be deployed around a disaster to capture environmental information at a variety of altitudes, real-time mapping of tweets and other data coming from a significant event, a notification application that pushes needs and directions to first responders via SMS, as well as a solution for connecting businesses with left over food with volunteers willing to deliver the food to those in need.
Just before lunch on Saturday, each team reported on exactly what they were intending to work on and what needs they had that other teams might be able to lend a hand on. Design and coding commenced, and by dinner many teams were able to stand and report on the functionality they had completed thus far. Many of the teams worked late into the night, and some even worked through the night. On Sunday the coding continued and at 4 pm each team took a turn presenting their work and talking about what they would do in the future. A panel of judges evaluated each presentation and the work of each team on its ingenuity, completeness, and a number of other factors and awarded prizes to the top teams. For a quick taste of RHoK Seattle, Johnny Diggz put together a video showcasing the weekend.
Having attended RHoK #3 in Seattle to see what the event is like, a few of our Engineers petitioned Bazaarvoice to sponsor RHoK in Austin. Now six months later: rhokaustin.org is ready to go. All the details for the Austin hackathon on December 2-4 are available on the site, and we’d love to have even more engineers, designers, HTML gurus, and project managers come to RHoK #4 in Austin. It’s going to be an amazing event and we hope to see you there!
Stop, collaborate and innovate: Engineering a company science fair
Many people are aware of Google’s “20 percent time,” and the number of innovations that have been produced because of it (Gmail, Google News, and AdSense to name a few). Several other companies have tried this before and after Google; one of the most famous examples of side-project innovation is 3M’s Post it Note. Software development companies are now emulating this model, holding hack-a-thons where flurries of innovation occur.At Bazaarvoice we’ve been holding regular Science Fairs every few months for well over the last year to allow our Engineering team to stretch their creative muscles. The format is simple; we schedule the two-day Science Fairs at times that are not likely to have a lot of conflicts with the regular release cycle, typically over a Thursday and a Friday. When the engineers arrive on Thursday, instead of working on the newest features in our evergreen platform, they get to choose what they want to work on. Teams of engineers (and designers) from across our Implementation, Development, Operations, and Support teams work together all day Thursday—often late into the night—and half of the day Friday to bring their ideas to life. After lunch on Friday the teams present their projects to a panel of judges and finalists, and winners are announced.
These types of events are an amazing addition to the company’s culture, and really help to drive new innovations into our products. Engineers walk away from the two days of coding with a new sense of empowerment, realizing just how much they can accomplish in such a short period of time. Additionally, many of the projects stretch the bounds of what our products can do already, showing us new areas for efficiency improvements, incredible new user experiences, or just plain fun new ways of looking at the same data.
Having done this for a year, we’ve learned a few things along the way about how to run a successful Science Fair. Here are my top three tips.
1. Get thematic
We’ve learned that providing some structure to the projects helps to give people direction. We’ve published a list of themes for the last two Science Fairs, and that has helped not only with the task of judging all of the projects by dividing it into manageable chunks, but has also been a way to help engineers discover projects that they are passionate about.
2. Don’t be afraid to show it off
Second, it is critical that participants be able to see as many of the projects as possible at the conclusion of the Science Fair. We still have room to improve on this, but including a Science Fair Open House the Monday after the fair provides everyone with a chance to see what their peers were able to accomplish.
3. Level set
Finally, it is worthwhile noting that you should not expect to get new features or products out of every event. It is much more valuable to keep the process very open and to allow everyone to enjoy themselves rather than feeling like they have to produce the next great thing.