Since the beginning of the year I have been quietly working on something new and it’s finally time to introducing it.
Web traffic data is critical for many businesses, but very few have a clue of what is really happening. When I say things like 80% of the traffic is made by robots and 50% are bad robots, people are surprised. However, this is something we commonly see in the wild. The key problem here is that the people building and operating websites are stuck with tools based on outdated paradigms.
That’s a shame, because when you look deeply at your web traffic, you can easily improve things like performance, reliability, security and costs. I founded Access Watch to fix that, analyzing web traffic data from anywhere and allowing our customers to see and touch something that was previously inaccessible.
But let’s go back to where it started.
In my spare time, I’m still operating websites, one of them being the venerable BlogMarks, a social bookmarking application that launched 15 years ago. One day, the performance of BlogMarks started to be problematic. We had a modest user base and were operating from a single server, but somehow we had enough traffic to bring the site down. How could that be? I dove into the data and this is what I saw:
The current experience for many website administrators: Access logs that are hard to read and impossible to decipher.
All I could see were Access Logs, lines of green text on a black background, that kept flowing. What could I use to make sense of this? Short answer: nothing.
So, I started to write code to analyze the data. What I got back was crazy: 90% of the traffic was made by robots, half of them bad, distributed over multiple IP addresses, disguising themselves as regular humans. With the insights from this tool I wrote, I was able to triage the traffic, block or slow down the most abusive agents. BlogMarks was now safe.
At this point, I began to realize that I was touching something important. I considered making a product out of it, but the vision just wasn’t clear enough yet.
This changed 2 years ago.
I was then busy building and operating the API at EyeEm, facing the same problem and many more: what’s really happening on our servers? Who’s abusing our platform? How are the apps using the API? How can we troubleshoot when something is wrong?
Like many other companies, we were using well known tools to process our data but that was not what was truly needed: it was still dumb pipes processing and displaying data.
Then I saw the light.
A cloud service, collecting web traffic data from anywhere, analyzing it in real time, getting more intelligent with the increasing volume of data, and delivering clear actionable insights back to the customer.
One night, I quickly hacked a prototype, plugged it on a few websites I operate and started to index the web traffic data. Mission accomplished! The database was growing quickly and the network effects instantly materialized: what was discovered on one website immediately benefitted the others.
Everyone building or operating websites will need this kind of service one day or another. It was a problem shared by millions of users, a scalable product, and an original position – I knew I was onto something.
Founding Access Watch
I knew I had to work on this product full time. Last January, I decided to quit my job at EyeEm and got early on the support from Christophe Maire and his team at Atlantic Labs. With former EyeEm colleagues, Alex and Johannes, we worked on a prototype of the user interface.
It started to look like something!
We now have a company running and are building the team. Last summer, Milan, a seasoned product designer, joined as Head of Design. We will soon be welcoming several new team members, and are always on the lookout for more talent. We’re hiring!
Now you know our story, let’s talk about yours.
In the coming weeks, if you want to help us (and let us help you), you have two opportunities.
- We made a first version of our product available as a WordPress plugin and you can install it in a few clicks. On your side, you’ll learn about your traffic and protect your website from abusing robots. From our side, this will help us train our algorithms and better identify agents, patterns and threats.
- If you have something that is more complex than a WordPress website (as many of you do!) get in touch with us. We want to know what you’re currently using to process your Access Logs, your current problems, your horror stories. With your feedback, we can make sure that we’re building the product you need. We should have something ready for you soon, the earlier you reach us, the earlier you’ll get access to it.
Don’t be shy! We can’t wait to have you on board soon, as customer or colleague.
To the future!