What databases do you use and why?
This post about the Uber engineering team's decision to switch from Postgres to MySQL raised some eyebrows. Here at Mode we're generally very happy using Redshift for the bulk of our analysis. Curious what databases y'all are using, especially for those folks who had a role in deciding which one to go with.
We use fivetran to load data from a few sources (Salesforce, Marketo, Desk.com, Postgres customer DB) into Redshift. We also pipe raw event data of dashboard usage for Parse.ly via our data pipeline product.
Redshift has been pretty good to us so far, but the inability to join on data from separate databases is pretty annoying given that fivetran by default loads data from each source to its own database. Haven't looked into whether we can update that or not.
We've also started messing around with BigQuery to query some of Parse.ly's network data and have been pretty amazed at the query performance you can get. BigQuery is also pretty awesome in that it provides a truly "ops-less" infrastructure. No clusters to think about or data schemas to consider. Just APIs that under the hood, take care of all of that for you. I think we'll eventually play around with BigQuery for our main data warehouse, but the switching costs are a bit too high for us right now.
If you're curious about the cloud SQL options for "big data", our CTO Andrew Montalenti recently wrote a pretty awesome post detailing the options out there and how they stack up http://blog.parsely.com/post/4516/cloud-sql-bigquery-redshift/.
I use AirTable for cloud database software - they're pretty good, particularly for small businesses. This article goes into greater detail about why they're great! http://fitsmallbusiness.com/best-cloud-database-software/
We use Skyvia to export all our data (from Salesforce, MailChimp, QuickBooks and some others.) in a BiqGuery and then we use the standard connector to connect our DWH with Mode Analytics. We use BigQuery because it has a significant amount of benefits:
- Super-efficient and completely cloud-native pricing (here is a cool blog post about their pricing https://cloud.google.com/blog/big-data/2016/02/understanding-bigquerys-rapid-scaling-and-simple-pricing)
- Easy Data Sharing (sharing as same as with google docs)
- Encryption (security on the top level)
- Customers (starting from Coca-Cola, Spotify and up to Disney).