26 Apr 16 RE: The New Gmail Meter
We recently added a new notification to our signup flow. In fact, many of you reading may have even been directed here straight from that very page. If so, then you know that we are keenly aware of the current state of our Gmail Meter report.
With some embarrassment, and a fair share of humility, we admit that our Gmail Meter script currently just does not work as intended. (If you’re curious of the reasons, we’ve previously written about our ongoing Apps Script problems.) Essentially, Gmail Meter has outgrown its existing Apps Script platform and will continue to fail as long as it remains on this platform.
Two months ago, back at the end of February, we announced that we had begun the substantial undertaking of building an entirely new Gmail Meter from the ground up. As typical with software development, we grossly underestimated just how much time this task would take. And for full disclosure, besides Gmail Meter, we here at ShuttleCloud have several projects with a couple large enterprises that take up much of our time.
I know that software development can sometimes seem like a bit of a black box, but this does not meant that the process can’t be readily understandable. With that said, we would like to update everyone on what we’ve been doing, or what’s taking us so long.
Lessons learned in software development
First though, let’s recap and take a retrospective on some of the lessons we’ve learned in 2016. Starting in about December or January, we began receiving an increasing number of emails reporting a series of error messages. While looking into the cause of these messages, we vastly improved our internal error reporting system. Doing so made us realizes that the error messages themselves were being sent by Google, and that we had absolutely no control over the content or the sending of these messages.
Furthermore, our improved reporting revealed that there were way more scripts failing than errors reported. There was no way at the time to explain what exactly was going on, so we just went ahead and added even more reporting. This time from a broader level, on how scripts were running and/or failing.
Upon analyzing this new data, we discovered that the legacy script was programmed to run every instance of Gmail Meter worldwide within the same 30 minute interval, every day. We immediately thought that this was the problem and implemented an algorithm that schedules all script instances to run evenly spread throughout the day.
That’s what we had originally thought, but the actual result? We ended up experiencing a drastic and dramatic increase in the number of error messages and number of scripts failing.
Sometimes pivoting means a fresh start
At our wit’s end, it was sometime around then that we started the conversation with Google representatives. These conversations only solidified our growing suspicion that in order to make any improvements to Gmail Meter at all, we actually had to rebuild the tool from the ground up. While this endeavor can be considered costly and risky, it can also be viewed as a bit of a blessing in disguise by allowing us the chance to fix a few of the problems with the legacy script that we never got right.
Chief among our concerns is – no surprises – the issue of scaling. The legacy Gmail Meter script has continued to fail at a very regular rate since the start of this year. Unfortunately, no amount of optimization will prevent this from occurring and would only result in more errors and/or script failures. With the new platform no longer on the Google Apps Script infrastructure, we will actually have control over how data is managed in the backend. This means that not only is the new report much more stable and error resilient, it is already much faster than the legacy script.
Additionally, we opted to remove several process heavy metrics, specifically the word cloud and subject line analysis sections of the legacy report. We made this decision after reviewing feedback you have provided us, ultimately realizing that these metrics were both costly and needed serious revision before they could bring actual value to our end users. For example, the word cloud for the legacy script did not exclude words from signatures and would thus consistently provide biased data.
This decision was also made in part because we are understanding of the sensitive nature of the data we have access to. We are committed to giving our users valuable data and insights while reducing the amount of identifying data that we need to process. If we have to prioritize other features first and are not able to do the analysis well, we would rather not do that analysis for the time being. Privacy and security is one of our main priorities for this new Gmail Meter.
New Gmail Meter design to look forward to
So what are some of the new features that we are prioritizing instead? Well, besides the focus on resilience and security, another one of our priorities is data accuracy. This refers to both validating the accuracy of the data and ensuring that the data is presented in an easily understandable manner. We’ve broken down just about every metric found in the legacy Gmail Meter and for the ones we’re keeping, we’ve also reevaluated how to display these metrics.
To give a bit more background on the legacy Gmail Meter script, Google Charts was used to generate all graphs within the report. While Google Apps Script is not directly compatible with Google Charts, Google maintained an App Script library that allows a script to create charts. The stipulation is that most of the Charts features available to Apps Script are within the UI service, which was deprecated in December 2014.
While the charts in the legacy Gmail Meter worked well enough, a major overhaul was bound to happen. We felt right now was the best time to fit in a redesign (and facelift) of both the email report and the web interface. Now that we’ll be using the full functionality of Google Charts, we’re considering other kinds of graphs – ones that can better represent the data accurately and understandably.
A redesign isn’t the only thing that we’ve been working on. There have been a few core features that have been coming up regularly in conversation with our users and with this new platform for Gmail Meter, we will be able to address some these long standing requests.