Thank you for offering this amazing service. Although current implementation of Analytics marginally covers my use cases, I felt obliged to subscribe for it in order to give something back and keep Netlify running. However, I do have in mind one suggestion that would make Analytics way more useful.
I think it would be very helpful for everyone if you would offer raw access logs export. This would provide the information required for any exotic use case and will temporarily cover all use cases until items in roadmap get implemented. Do you want to compare custom date ranges or get stats for individual articles? Download the raw server logs and use AWStats or GoAccess or Scratchy or your favorite logs analyzer or even your own script!
Various download options may be implemented, like for example one file per month, or last month plus month-to-date, or only year-to-date, or only rolling 30-days⌠Any option would work if someone would regularly download the generated files. You are in a better position to judge the effort required and other variables for each option and to choose what to implement. Now, if you are thinking to support logs exporting with custom date ranges, or with API endpoints, that would fill me with great joy.
Offering raw logs exports for Analytics customers is not competitive to the Analytics product, rather a supplement to it. Logs exports offer the option to do further analysis than what provided by the predetermined charts. My opinion is that log exports will make the Analytics product more lucrative. Imagine the next time youâll answer a support request instead of âWe canât do this right nowâŚâ with âWe canât do this right now, but you can downloadâŚâ; which sounds more appealing?
I know that youâre already gathering the data in order to analyze them and create the current reporting. Isnât this feature a low hanging fruit that would worth prioritizing before implementing the rest of the Analytics roadmap?
Thank you for hearing this and again thank you for the great service you offer.
Howdy & thanks so much for the thoughtful feedback!
We have put a lot of consideration into offering raw log download, and it is not practical for us at present. As a start, we are not permitted to give you any personally identifying information about your visitors, such as IP address and in some cases user agent due to the GDPR and other privacy laws, so the logs would need to be scrubbed first, at a minimum.
I have added your voice to the open feature request asking for this from some other customers, but I do not think it is likely to happen soon. Your best bet for getting better analytics information is asking for analytics to be improved, rather than asking for raw access logs - since we are still actively working on the analytics product.
Any feedback on that topic (improving analytics without offering raw access logs) would be most welcome and I would personally review it with the analytics development and project management team, so you can be sure it would be heard if you wanted to provide it
@fool it kinda feels like an excuse not to give them and enforce users to register for analytics, and Iâm all for GDPR. Itâs like AWS saying we can host your S3 buckets but we canât give you access to the logs. If I host my website at Netlify, I should be able to get the logs.
Right now, not having raw server access logs means I either have to add a JS version like Google Analytics and add trackers on my website, or pay Netlify (which I donât mind) specifically for that feature and have a very limited version of analytics.
Love Netlify and the service you guys built, but this is the kind of move which makes me think maybe I should use Gitlab/Github pipelines and just upload my static website to an S3/Cloudfrout bucket, get access logs and use Awstats.
I understand your feelings, but this was our policy for years before we had an analytics product too, so your accusation while reasonable to guess at, is not true.
Since my post almost a year ago, log export is closer to being implemented - we are talking about how to do it scalably and sustainably, taking into account privacy laws. I donât have any proposed timeline for completion, nor can I guarantee it will be available at all account levels, but your use case is not weird and we do want to solve it, with or without our analytics product. It is, however, a very hard problem (we have terabytes of access logs PER DAY), so there is no quick solution.
Of course you need to make the decision that is correct for your business. If building your own Netlify (thereâs a heck of a lot more to it than S3 + Pipelines if you want to do it well) is the only path you see, I will advise you to follow that path, because your businessâ needs are real and immediate, regardless of what service it is possible, convenient, or planned, for us to provide.
Hope you can stay with us, but please do NOT take this as a promise that this problem will be solved for you in the way you want on any timeline, so you need to make the correct choice for yourself.
It would be awesome if you can provide server logs, because your Analytics itâs nice but not sufficient from an SEO perspective: how do we track Googlebot hits (something basic), for instance? There are many insights we can get through server logs that we are now missing⌠Hope you can make some kind of move regarding this matter.
@javier ist right. I also came here because of this missing feature with regards to GoogleBot Crawler. There are tools out there which you feed the access.log and it will give you GoogleBot Crawler Stats. Very much needed from SEO perspective. Not having this is a real shortcomming and GDPR cant be an excuse. Anonymizing IPs before making them available shouldnt be too hard right?
Hi, I also am interested in something closer to the raw logs.
My particular use cases:
I have a few services I use for testing uptime and page load speeds, and I currently have no way to exclude them when looking at traffic. Likewise for excluding crawlers.
I have a product I sell exclusively in the US. Seeing all traffic in the US lumped together is effectively useless for me in terms of marketing efforts. Based on my limited understanding of the California privacy laws (and general âpersonal informationâ laws), reverse geocoding IPs is not considered personal information so long as itâs coarse enough data. When I was an iPhone developer, we provided location data limited only to metro area. Being able to see traffic by metro area, or at least by state, would be immensely helpful.
As a matter of principle, I would like to limit use of granular tracking services and tracking cookies. I would like to see Netlifyâs offerings meet those needs.
Nearly 2 years on from the OP, it seems that netlify isnât really going in any direction. Doesnât seem to improve at all with any of the featuresd requested.
However, I do have in mind one suggestion that would make Analytics way more useful.
