We were pleased to see the renewed debate between “manual instrumentation” and “auto-capture” solutions in recent blog posts from product analytics vendors, Amplitude (Auto-track is [Still] Bad) and FullStory (Why autocapture is the future of Product Analytics). As a company dedicated to transforming digital experience data capture into a fast, easy and automatic process – across all use cases and platforms – we certainly have a horse in this race!
It won’t surprise anyone that we agree with the main premises of the FullStory post:
- With manual instrumentation, most events are never gathered at all. These “long tail” events – all the other actions that users take that you will never manually instrument – are often the ones that provide the most important input for discovering valuable insights. In other words, you can’t uncover unknown unknowns without access to all the data.
- Maintaining data integrity as products evolve is easier with auto-capture; with manual instrumentation, you start losing data the moment an event breaks.
- Comprehensive data capture powers a privacy-first approach.
- It makes sense to manually instrument a small subset of events that serve core, consistent KPIs.
Beyond these good points made by FullStory, it is important to drill down into some of the specific issues raised by the recent Amplitude post (as well as an earlier post of theirs) – the other side of the debate – because technology is marching on, and there is much more here than initially meets the eye.
“Auto-tracking doesn’t eliminate work. It shifts work to a less scalable process… auto-tracking SDKs collect events from page views, clicks, and object interactions and assign them generic names like page viewed, button clicked, or form submitted.”
While it is true that some previous auto-track technologies assign generic (and thus useless) names to events, Convizit delivers the vast majority of events already named, enriched with relevant properties, structured and ready for immediate use. Website owners do not need to invest any work in selecting, defining, tagging, coding or structuring events in order to make their data available or useful.
“So while auto-tracking might save a few hours of initial work…”
From discussions with hundreds of companies, it is clear that the amount of time and work involved in implementing manual tracking solutions is typically measured in weeks or months. So, Convizit’s intelligent auto-capture solution is not saving “a few hours of initial work,” but rather much more than that.
“Every time a new product development rolled out or a user interacted with a new part of the product, new pages needed to be manually tagged so it could be recognized.”
This statement does not apply at all to Convizit’s approach, although it is absolutely correct regarding earlier auto-track solutions that require every new website element to be manually selected, and identified using CSS selectors, before interactions with them become available. Convizit automatically collects, structures and delivers every new event, with no manual tagging required by the website owner before the data is available for any use case.
“Auto-tracking Misses the Full Story of What’s Happening in Your Product… A lot of the most valuable and relevant information actually exists in properties rather than events.”
It is absolutely correct that including rich event properties with events is critical for event data to be valuable. And it is true that no auto-track solutions prior to Convizit automatically collected event properties (manual tagging was still required when using them). One of Convizit’s key advantages is that our technology automatically collects and structures extensive event properties for every user event. So, companies using Convizit get the best of both worlds – very rich event properties for every event, with no manual work required.
“Properties are not auto-tracked, so if you want to capture them you’ll need engineering support that you sought to avoid in the first place”
As mentioned in the previous point, this is incorrect regarding Convizit. All relevant properties available in the page are automatically captured, with no engineering support required.
In cases where a desired event property does not exist on the page, Convizit supports “custom properties” so that you can add the occasional such data point.
“With property tagging and weekly admin maintenance considered, the time-to-value argument for auto-tracking starts to fall apart.”
Exactly the opposite applies to companies using Convizit: zero event or property tagging (or maintenance) means that Convizit’s rapid time-to-value is unprecedented, both during initial deployment and during ongoing use.
“Auto-tracking Breaks… relying on the consistency of their CSS. … Eventually a code change breaks tracking”
This claim is correct regarding previous auto-track technologies – because they rely on CSS selectors to identify website elements. In these cases, it is absolutely correct that auto-tracking breaks frequently, because the structure/design/coding of webpages on which they rely changes frequently.
On the other hand, Convizit’s AI-based solution takes a completely different approach to automatic event tracking that completely eliminates this major weakness of CSS selector-based solutions. Because Convizit identifies webpage elements based on their use, meaning and context, ongoing website changes do not break tracking. Convizit’s unique element identification and grouping technology deliver an unprecedented degree of tracking continuity and reliability.
“…when used on anything beyond a simple web app that’s rarely updated, auto-tracking tends to break. This is because auto-track solutions are built for marketing websites, which are much more static than products.”
This statement does not apply whatsoever to Convizit, which was developed from the ground up to handle complex e-commerce sites, SaaS apps, single-page applications (SPAs), React-based websites and so forth.
“Using auto-track approaches to tagging is a step backwards to the DOM scraping days and brings back a lot of its fragility.”
This repeats the same point mentioned above – about relying on CSS selectors to identify website elements – in different words. Yes, earlier auto-track solutions suffer from fragility and data loss as a result. As explained above, Convizit’s completely different AI-based paradigm completely eliminates this major weakness.
“Auto-track products inherently collect too much data and make it difficult to find the meaningful data you need to be successful.”
The only way to fully analyze and understand user journeys (and to comprehensively address many other analytic and marketing use cases) is to have access to all user activity. However, this is only practical when this data is automatically collected, structured and immediately usable; when every individual event needs to be manually selected, tagged, coded and maintained, then one is forced to select only some events – and missing lots of data means missing lots of insights. (Of course, for use cases in which only certain events are relevant, Convizit makes this fast and easy to accomplish, as well.)
“Auto-track Caused Security Incidents Where It Accidentally Captured Sensitive Information”
Convizit was built from the ground up with “privacy by design” and implements multiple automatic and manual layers to prevent the capture of PII. For example, the content of text input boxes is never collected by default. Furthermore, Convizit makes it very easy for customers to exclude any specific webpage elements that they wish (customers can do this in three different ways, two of which utilize a friendly UI).
“What if I Forgot Something? … In most cases, it is ok to add some new tags and wait for a few weeks to get enough data to answer your business question”
One of the biggest advantages of any auto-track solution is that product analysts, marketers and other business users do not need to decide, in advance, what data points to collect. All the user activity data they might ever need is readily available. The above statement is a great illustration of the old way of looking at digital user experience data capture; in today’s fast-moving business environment, no analyst or marketer would be happy waiting “a few weeks” to get the data they want now! The fact that manual tracking tools require this delay (which involves getting a developer to add code to the site and then wait to collect the data from the site) is a huge disadvantage of manual tracking solutions.
“Why is it so difficult for them to plan their implementation ahead of time?”
It makes sense that companies selling a manual tracking solution – an approach that requires lots of planning and implementation in order to get started – insists that planning needs to occur before implementation. The beauty of Convizit is that complete event data begins flowing within days, effortlessly, allowing customers to plan their initial analytics in parallel to the start of actual data collection, or afterwards. Furthermore, for many use cases, building an analytics plan around what the data tells you – and not the other way around – allows far more intelligent and valuable results. As the old saying goes, you often don’t know what you don’t know. Any a priori tracking plan could be missing many of the most important data points, and you might never even know it.
“The data is very difficult to export to other systems… when you export the data to a warehouse—or wherever you’d like to send it—you’re going to get a bunch of raw data that isn’t named what you’ve named it in the tool.”
This does not apply whatsoever to Convizit. Convizit was designed exclusively to capture digital experience data and deliver it to other systems. All events are delivered pre-named, pre-structured and automatically enriched with all available event properties.
“Imagine being able to “auto-track” all of the data you might need from your customers without having to spend time doing solution architecture, design or begging developers to build a data layer and set analytics events and properties. Who wouldn’t want that?”
Yes! This is exactly what Convizit is delivering to the market, for the first time.
It is natural for there to be resistance to changing approaches and adopting new methodologies, especially in a decades-old field, such as manual event instrumentation. But the days of collecting website activity data the old way – by manually tagging/coding each individual event and property – are winding down.
Addressing all of the auto-capture challenges mentioned above requires solving several extraordinarily difficult technical problems. In fact, finding solutions for these technical challenges may not have been possible before the era of machine learning and advanced AI. But this is exactly what we have done – applying new approaches and new technologies to usher in the next generation of digital behavior capture.
We invite you to contact us for a free trial and see for yourself how much time, effort and frustration this next-generation solution will save you.