Upcoming change that will affect stats

BaldyBaldy Registered Users, Super Moderators Posts: 2,853 moderator
edited March 17, 2015 in SmugMug Product News
Hey everyone,

Possibly as soon as the first of July, we’re going to make a major change in the way images and videos are delivered, and we thought we’d give you some advanced notice because it affects how stats are reported.

Your images and videos are stored at Amazon, but we have used Akamai for years to deliver photos and videos from Amazon to you. Akamai distributes your images via tens of thousands of servers around the world, keeping frequently requested images on a server near you for faster delivery.

But now we’re moving away from Akamai to another provider. We’re doing it for several reasons, chief among them being much faster and smoother delivery of video, and more advanced security. We're hopeful, and our testing seems to show, that image delivery should be somewhat faster for most people depending on where they live. It's possible that we may see somewhat slower delivery for some people because Akamai had great worldwide coverage.

But in a world of trade-offs, we’re losing the ability to discern between user views of your photos, and visitor views. The way we had been working with Akamai, we could detect when you were browsing your own photos and not count them in the stats.

We’re very sorry we won’t be able to do that anymore when we make this switch, and for some people subtracting out owner views was a great feature, but this is a tradeoff we felt we had to make for the speed and security gains we’re seeing in our testing.

Let us know if you have questions!

Thanks,
Baldy
«1

Comments

  • AllenAllen Registered Users Posts: 10,013 Major grins
    edited June 20, 2014
    This sounds like stats will become completely useless then? Or at least for anything the user has touched?
    Al - Just a volunteer here having fun
    My Website index | My Blog
  • BaldyBaldy Registered Users, Super Moderators Posts: 2,853 moderator
    edited June 20, 2014
    Allen wrote: »
    This sounds like stats will become completely useless then? Or at least for anything the user has touched?
    Well I think the normal use case is one view comes from you, the rest from your visitors, because your browser caches the images you've already viewed and doesn't fetch them from us. If you were to also view your images on a mobile device, then two views from you.

    The use case I'm familiar with is a photographer puts boudoir photos in a gallery, then sees there was a view or two and emails the heroes to find out who it was, but it was them. We'd love to solve for that use case like we did before, but tradeoffs.
  • denisegoldbergdenisegoldberg Administrators Posts: 14,376 moderator
    edited June 21, 2014
    Baldy -
    Thanks for the heads up.

    --- Denise
  • phaserbeamphaserbeam Registered Users Posts: 452 Major grins
    edited June 24, 2014
    What about the slideshow-widget?
    If i run the slideshow on my own computer... will the photos counted as views?
  • BaldyBaldy Registered Users, Super Moderators Posts: 2,853 moderator
    edited June 25, 2014
    We have a major and unexpected update: in the middle of the night Pacific Time we tested CloudFlare, our new content delivery network, thinking the test would last a short time and we'd evaluate performance and look for bugs.

    But it went so well and our Euro heroes reported such great results for both images and video (especially video) that we haven't yet stopped the test. So as of 9:15 Pacific Time, it's live with your images and videos. We can switch back in an instant if we find an issue.

    Let us know if you see any problems but so far, the help desk has been silent about it.
  • BenBen Vanilla Admin Posts: 513 SmugMug Employee
    edited June 25, 2014
    phaserbeam wrote: »
    What about the slideshow-widget?
    If i run the slideshow on my own computer... will the photos counted as views?

    Yes, the first view of each photo should count as a view. But if you left it running for days straight, your browser would (in theory) have cached the photos and only that first view would matter. So no matter how many times the slideshow cycled, you would have only generated a single view on each photo.

    Does that make sense?
    Smug since 2003
  • AllenAllen Registered Users Posts: 10,013 Major grins
    edited June 25, 2014
    I have all my bird videos in a smart gallery (keyword bird_videos) and they seem to
    be loading much faster. thumb.gif
    http://www.photosbyat.com/MyKeywords/Bird-Videos

    The only nit is some (gallery view) run pixelized, not the highest resolution. In the caption I have the
    the lightbox view link which shows a much larger video. But it seems I do not get a good resolution
    without manually changing the size. Any thoughts? I'm afraid visitors will never see this size changing.

    One other thing, it seemed like none of videos where cached and had to reload from start each re-run.
    Al - Just a volunteer here having fun
    My Website index | My Blog
  • BaldyBaldy Registered Users, Super Moderators Posts: 2,853 moderator
    edited June 25, 2014
    Allen wrote: »
    I have all my bird videos in a smart gallery (keyword bird_videos) and they seem to
    be loading much faster. thumb.gif
    http://www.photosbyat.com/MyKeywords/Bird-Videos

    The only nit is some (gallery view) run pixelized, not the highest resolution. In the caption I have the
    the lightbox view link which shows a much larger video. But it seems I do not get a good resolution
    without manually changing the size. Any thoughts? I'm afraid visitors will never see this size changing.

    One other thing, it seemed like none of videos where cached and had to reload from start each re-run.
    Thanks for pitching in, Allen. Checking.
  • BenBen Vanilla Admin Posts: 513 SmugMug Employee
    edited June 25, 2014
    Allen wrote: »
    I have all my bird videos in a smart gallery (keyword bird_videos) and they seem to
    be loading much faster. thumb.gif
    http://www.photosbyat.com/MyKeywords/Bird-Videos

    The only nit is some (gallery view) run pixelized, not the highest resolution. In the caption I have the
    the lightbox view link which shows a much larger video. But it seems I do not get a good resolution
    without manually changing the size. Any thoughts? I'm afraid visitors will never see this size changing.

    One other thing, it seemed like none of videos where cached and had to reload from start each re-run.

    Yeah, videos in particular should get much faster. We put a LOT of work into the video streaming architecture in this switch, though it is a tough problem and we are definitely keeping an eye on performance. I really appreciate the feedback. :)

    The auto-sizing should work a lot better, but we have a bunch of a improvements we are working on for it. For one, we have improved the way we encode videos for our new architecture, so newer videos will likely snap to a higher resolution quicker/better. But even for older videos we can do a lot better. We have improvements in the works for the video player to do a better job of getting you to a higher resolution quicker. We had a deadline with the CDN switch, so the video player improvements weren't a requirement for this change. They are coming though. :)

    Regardless of other improvements coming, right now videos should be significantly faster and more reliable than they were yesterday. The resolution changing should also be quicker and more reliable, we just want to make it even MORE quick and reliable. thumb.gif

    As for caching, I know our engineers/ops were chatting with CloudFlare late into the night last night on making sure videos get cached properly at local nodes. I don't know the status of that unfortunately, but it is also actively being looked at.

    Thanks again, I really appreciate the feedback on this. It was not a trivial change. :)
    Smug since 2003
  • BenBen Vanilla Admin Posts: 513 SmugMug Employee
    edited June 25, 2014
    As a quick update, I just heard that there is some issue getting the stats from CloudFlare right now. Our engineers and ops are scrambling to sort the issue, but you might see some delayed results this morning.

    I honestly don't know anything more than that, but I will update you guys if I hear anything good or bad.

    Don't worry, we *definitely* understand how crucial stats are to you guys.
    Smug since 2003
  • AllenAllen Registered Users Posts: 10,013 Major grins
    edited June 25, 2014
    One thing I noticed in the past were some videos in lightbox would open very large, some beyond browser
    width. Changing size would vary video size. I have not seen this for a few years? though. Looks like all here
    now open in lightbox at ~930x525 no mater what resolution is selected so missing the large HD size.
    24" monitor FF. Selecting 480 > 1080p HD all same size.

    Edit: lightbox is still much larger then gallery view so really miss the click to lightbox for videos.
    Adding /A to the link for me helps but visitors don't have a clue.
    Al - Just a volunteer here having fun
    My Website index | My Blog
  • shandrewshandrew Administrators, Vanilla Admin Posts: 33 SmugMug Employee
    edited June 25, 2014
    Hi Allen, currently there's no easy way to click to get from a video to lightbox (from the gallery view), though it's certainly something that I wish SmugMug had--but there's a difficult tradeoff as most expect play/pause behavior from clicking on a video.

    What are you using to get to lightbox? The only way I'm aware of from a video gallery view is with keyboard shortcuts: http://help.smugmug.com/customer/portal/articles/84540-are-there-keyboard-shortcuts-for-smugmug-

    If you type "a" you'll get the screen-filling resolution (the same /A you get for lightboxes when clicking on a gallery image)
    I work at SmugMug but these opinions are usually my own.
  • AllenAllen Registered Users Posts: 10,013 Major grins
    edited June 25, 2014
    shandrew wrote: »
    Hi Allen, currently there's no easy way to click to get from a video to lightbox (from the gallery view), though it's certainly something that I wish SmugMug had--but there's a difficult tradeoff as most expect play/pause behavior from clicking on a video.

    What are you using to get to lightbox? The only way I'm aware of from a video gallery view is with keyboard shortcuts: http://help.smugmug.com/customer/portal/articles/84540-are-there-keyboard-shortcuts-for-smugmug-

    If you type "a" you'll get the screen-filling resolution (the same /A you get for lightboxes when clicking on a gallery image)
    Adding a /A behind the gallery browser link and LB opens. But as you say "a" works, have to remember that.

    My hand is on the mouse and the other is usually away from the keyboard so hitting "a" is an extra
    movement. But why has Smug stolen the cntl and alt "a" makes no sense? I used to use alt-a to
    select everything to check for hidden page items. Good for troubleshootin'.

    ... and visitors have no clue the "a" key works. But they are use to clicking a photo.
    Al - Just a volunteer here having fun
    My Website index | My Blog
  • AllenAllen Registered Users Posts: 10,013 Major grins
    edited June 25, 2014
    Hope they fix the start up resolution soon, "auto" is not working, getting lowest resolution and no way to set it without starting video.
    Al - Just a volunteer here having fun
    My Website index | My Blog
  • shandrewshandrew Administrators, Vanilla Admin Posts: 33 SmugMug Employee
    edited June 25, 2014
    Allen wrote: »
    Hope they fix the start up resolution soon

    Yeah! I agree completely. Now that the streaming video delivery is faster and way, way more consistent, getting some smart logic into picking startup resolution should be a lot easier.
    I work at SmugMug but these opinions are usually my own.
  • mbonocorembonocore Registered Users Posts: 2,299 Major grins
    edited June 26, 2014
    Thanks for the feedback everyone....Allen, we will keep you posted.
  • livverlipslivverlips Registered Users Posts: 31 Big grins
    edited July 9, 2014
    Baldy I am one of the users who who now finds the stats feature completely and utterly useless, and I am quite upset about this removal. I just spent the better part of two hours trying to figure out exactly how two private unlisted password protected galleries had over 600 image views each. After changing many of my personal passwords, I emailed multiple clients to make sure they did not share the gallery links or images. I finally relented and emailed support and got this news. To say the least I am now very embarrassed having had to ask my clients if they had shared the galleries. If there was a notice sent out about the removal of BASIC functionality of the stats it should have been much BIGGER AND LOUDER! prior to its implementation!

    Since I often have to edit in between uploads and am uploading multiple versions of the same images I have to review my galleries on multiple platforms as I post and make changes. Everything is now skewed and pointless. Before I could share a private gallery to a client or to someone I trusted and know exactly when it has been viewed. I could email images to a potential business partner and know that they did in fact see specific images. I could tell exactly which images were the most popular in a quick glance and see what size images folks were being drawn to for a certain gallery. I could manage and garner how much traffic I am directing from my posts on social media, simply by viewing the days traffic after I make a post of a link to an unlisted gallery that had not been shared elsewhere yet. Now it's all completely useless! As a member since 2008 this is a feature I would say I'm the most upset to see go. I know I can start using external traffic tracking and have already set up an account at statscounter but for as much as I am paying a year for you to host my photos I don't think I should have to go outside my service to another website to get ACCURATE image view stats. I also don't know exactly what kind of detailed information they will provide for specific image views.

    This could very well be a dealbreaker for me. Since this was just implemented and I was completely unaware of this removal until today I'm quite positive you will be getting a lot of feedback from users who like me relied heavily on the stats feature. Frustrated and saddened by this.


    Baldy wrote: »
    Hey everyone,

    Possibly as soon as the first of July, we’re going to make a major change in the way images and videos are delivered, and we thought we’d give you some advanced notice because it affects how stats are reported.

    Your images and videos are stored at Amazon, but we have used Akamai for years to deliver photos and videos from Amazon to you. Akamai distributes your images via tens of thousands of servers around the world, keeping frequently requested images on a server near you for faster delivery.

    But now we’re moving away from Akamai to another provider. We’re doing it for several reasons, chief among them being much faster and smoother delivery of video, and more advanced security. We're hopeful, and our testing seems to show, that image delivery should be somewhat faster for most people depending on where they live. It's possible that we may see somewhat slower delivery for some people because Akamai had great worldwide coverage.

    But in a world of trade-offs, we’re losing the ability to discern between user views of your photos, and visitor views. The way we had been working with Akamai, we could detect when you were browsing your own photos and not count them in the stats.

    We’re very sorry we won’t be able to do that anymore when we make this switch, and for some people subtracting out owner views was a great feature, but this is a tradeoff we felt we had to make for the speed and security gains we’re seeing in our testing.

    Let us know if you have questions!

    Thanks,
    Baldy
  • livverlipslivverlips Registered Users Posts: 31 Big grins
    edited July 9, 2014
    I don't believe this is accurate as I know in one gallery that the client only viewed the images literally once. so in a gallery of 80 images 80 of the views were from a client and over 500 image views ended up being from myself as I was posting, editing, arranging and adding new images, prior to sharing with the client. It is counting my views multiple times on the same image. Also your premise relies on an assumption that users never clear their cache.
    Baldy wrote: »
    Well I think the normal use case is one view comes from you, the rest from your visitors, because your browser caches the images you've already viewed and doesn't fetch them from us. If you were to also view your images on a mobile device, then two views from you.

    The use case I'm familiar with is a photographer puts boudoir photos in a gallery, then sees there was a view or two and emails the heroes to find out who it was, but it was them. We'd love to solve for that use case like we did before, but tradeoffs.
  • mbonocorembonocore Registered Users Posts: 2,299 Major grins
    edited July 10, 2014
    livverlips wrote: »
    Baldy I am one of the users who who now finds the stats feature completely and utterly useless, and I am quite upset about this removal. I just spent the better part of two hours trying to figure out exactly how two private unlisted password protected galleries had over 600 image views each. After changing many of my personal passwords, I emailed multiple clients to make sure they did not share the gallery links or images. I finally relented and emailed support and got this news. To say the least I am now very embarrassed having had to ask my clients if they had shared the galleries. If there was a notice sent out about the removal of BASIC functionality of the stats it should have been much BIGGER AND LOUDER! prior to its implementation!

    Since I often have to edit in between uploads and am uploading multiple versions of the same images I have to review my galleries on multiple platforms as I post and make changes. Everything is now skewed and pointless. Before I could share a private gallery to a client or to someone I trusted and know exactly when it has been viewed. I could email images to a potential business partner and know that they did in fact see specific images. I could tell exactly which images were the most popular in a quick glance and see what size images folks were being drawn to for a certain gallery. I could manage and garner how much traffic I am directing from my posts on social media, simply by viewing the days traffic after I make a post of a link to an unlisted gallery that had not been shared elsewhere yet. Now it's all completely useless! As a member since 2008 this is a feature I would say I'm the most upset to see go. I know I can start using external traffic tracking and have already set up an account at statscounter but for as much as I am paying a year for you to host my photos I don't think I should have to go outside my service to another website to get ACCURATE image view stats. I also don't know exactly what kind of detailed information they will provide for specific image views.

    This could very well be a dealbreaker for me. Since this was just implemented and I was completely unaware of this removal until today I'm quite positive you will be getting a lot of feedback from users who like me relied heavily on the stats feature. Frustrated and saddened by this.

    I apologize for the confusion regarding this on your account. Can you please send me a link to the specific gallery you are referring to? Please feel free to PM it to me.

    Thanks you.
  • Tan68Tan68 Registered Users Posts: 9 Beginner grinner
    edited July 10, 2014
    I think the old way of recording views would be useful to me.
    The new way isn't a deal breaker but it is a consideration.
    The old way was useful to me in the way you describe using 'stats'.
    The new way adds nothing over other services.
    I am trying to decide on service, now.
    Cost for Smugmug isn't in annual fee. It is in percent per earnings.
    livverlips wrote: »
    I don't believe this is accurate as I know in one gallery that the client only viewed the images literally once. so in a gallery of 80 images 80 of the views were from a client and over 500 image views ended up being from myself as I was posting, editing, arranging and adding new images, prior to sharing with the client. It is counting my views multiple times on the same image. Also your premise relies on an assumption that users never clear their cache.
  • livverlipslivverlips Registered Users Posts: 31 Big grins
    edited July 10, 2014
    mbonocore wrote: »
    I apologize for the confusion regarding this on your account. Can you please send me a link to the specific gallery you are referring to? Please feel free to PM it to me.

    Thanks you.

    Sending you one of the recent links now, but the one that had client views has had the images deleted since they were just a copy for approval. The other gallery has a similar amount of image views. I still have it password protected though, and it shouldn't have had any public views. However since Ive had an image "featured" while its been protected anyone can use the thumbnail image of that password protected image to bring up the other size images (the owner has made available) of that particular photo just by changing the html link. I'm fairly certain that no other images can be accessed though by the public.
  • mbonocorembonocore Registered Users Posts: 2,299 Major grins
    edited July 12, 2014
    livverlips wrote: »
    Sending you one of the recent links now, but the one that had client views has had the images deleted since they were just a copy for approval. The other gallery has a similar amount of image views. I still have it password protected though, and it shouldn't have had any public views. However since Ive had an image "featured" while its been protected anyone can use the thumbnail image of that password protected image to bring up the other size images (the owner has made available) of that particular photo just by changing the html link. I'm fairly certain that no other images can be accessed though by the public.

    I have the link and I am investigating. Any further galleries you experience this with, please send over to me. Thanks!
  • mbonocorembonocore Registered Users Posts: 2,299 Major grins
    edited July 12, 2014
    Hey guys,

    Just an FYI, I have a conversation going with my Stats developer and hoping to get some more detailed information as to how/when stats are counted. I personally haven't tested this myself, nor do I ever look at my stats, so I can't really give a good explanation, but I am working on it for you.

    Will update you all as soon as I can, most likely sometime on Monday afternoon/evening.
  • RichardRichard Administrators, Vanilla Admin Posts: 19,962 moderator
    edited July 13, 2014
    mbonocore wrote: »
    Hey guys,

    Just an FYI, I have a conversation going with my Stats developer and hoping to get some more detailed information as to how/when stats are counted. I personally haven't tested this myself, nor do I ever look at my stats, so I can't really give a good explanation, but I am working on it for you.

    Will update you all as soon as I can, most likely sometime on Monday afternoon/evening.
    I'd be interested in how pics linked on Dgrin are counted. It looks like there must be some caching going on, as SM regularly reports fewer hits than would correspond to Dgrin page views. This is not new behavior, BTW. What does seem to be new is that there looks like an increase in hits counted on SM that don't correspond to page views reported by Statcounter or Google Analytics. I am seeing this in galleries that I know I haven't accessed myself. Even stranger, in one case I saw a dozen hits on one gallery reported in Statcounter but none at all in SM. headscratch.gif
  • Jerry Latta PhotographyJerry Latta Photography Registered Users Posts: 17 Big grins
    edited July 21, 2014
    Possibly Smugmug engineers or whoever are really busy working on more important items. Maybe they are on the brink of something above and beyond, I have no idea. But I do know that the statistics are totally wrong. I am one of the people who does look at the stats, but when they are totally inaccurate, they are totally worthless. I had reported it and was told that it was being worked on and I sent an email to a person here and never heard anything. Each time I gave examples of why I was questioning the stats. Today I have another great example. I uploaded 112 images to a gallery that needed a password to gain access. I sent the link to my daughter, her husband, and my wife so that they could see them. My daughter and my wife looked at them and I guess since I uploaded them (without viewing individual images) I think I would also count as having viewed each image. I uploaded the images a couple of hours ago and just looked at the statistics and there have been 1901 views – around 17 views per image uploaded. Individual views ranged from a high of 28 to a low of 7. I just called my daughter to see if she had forward the link and was told “No” so I asked if she had looked at them several times and again the answer was “No”. I asked about her husband and she said that he had not yet been home so he would not have seen them at all. I asked my wife and she said that she had looked at all of them once and several a second time on her iPad. So basically there should be around 400 (maximum) views of the gallery that has 1901 view. Maybe the employees at Smugmug just really love my images and have hacked into the gallery and racked up 1500 views.

    I guess I am saying that as of right now and for some time in the past, the statistics are not accurate at all. In the past I had always been impressed with the rapid replies from Smugmug and their fixing problems. Currently that isn’t happening.

    Jerry
  • phaserbeamphaserbeam Registered Users Posts: 452 Major grins
    edited July 23, 2014
    I don't have much visitors... i never had and i don't need to. I also do not take care about stats... and i don't know how good the SM stats can be compared with Google Analytics... this is how it does look like here, see screenshot. Is this "normal" ? Especially the counts?
  • thenickdudethenickdude Registered Users Posts: 1,302 Major grins
    edited July 24, 2014
    phaserbeam wrote: »
    I don't know how good the SM stats can be compared with Google Analytics

    They can't be compared at all. One counts image views, the other pageviews.
  • phaserbeamphaserbeam Registered Users Posts: 452 Major grins
    edited July 24, 2014
    They can't be compared at all. One counts image views, the other pageviews.

    OK, thx for the reply.
  • AlamethAlameth Registered Users Posts: 94 Big grins
    edited July 24, 2014
    This change disproportionately impacts small low-volume sites like mine. You've basically given me a choice between having stats and being able to do site maintenance.
  • BaldyBaldy Registered Users, Super Moderators Posts: 2,853 moderator
    edited August 6, 2014
    livverlips wrote: »
    Baldy I am one of the users who who now finds the stats feature completely and utterly useless, and I am quite upset about this removal. I just spent the better part of two hours trying to figure out exactly how two private unlisted password protected galleries had over 600 image views each. After changing many of my personal passwords, I emailed multiple clients to make sure they did not share the gallery links or images. I finally relented and emailed support and got this news. To say the least I am now very embarrassed having had to ask my clients if they had shared the galleries. If there was a notice sent out about the removal of BASIC functionality of the stats it should have been much BIGGER AND LOUDER! prior to its implementation!
    Hi Livverlips and everyone else we affected with this,

    First of all, I'm very sorry this caught so many people by surprise. I'm afraid I'm the culprit because the engineers warned me ahead of time that the downside of our new image delivery network is not being able to count owner views. I thought that would be okay given the performance gains, not thinking through all the ways owners could rack up big, unexpected image views and the confusion it could cause.

    Here's a common scenario: you create a new gallery and make it private. You upload 300 images using our uploader. You click Done and it dumps you into the gallery. It's only natural that most people wouldn't think of the Done button as generating a lot of image views, but suddenly we just served up a lot of images to you. How many? Well that depends first on the gallery style. If it's one of the styles that puts all images on a single page, it could be a lot, depending on whether you scroll and the width and resolution of your monitor.

    Adding to the count, we do our best to predict which images you're most likely to click, so we deliver those to your browser too. They aren't visible to you until you click on a small image and get a bigger one, but to us it's an image we sent to your browser and for better or worse, we count it. We do this because we want you to see it instantly and be pleased by the speed.

    Sometimes we get emails or see posts showing crazy image views, wondering what kind of bugs we have. We've chased a fair number of these down and, so far, we really are reporting images delivered to the browser, it's just that they are coming in unexpected ways like I outlined above.

    At this point you're probably thinking, what kind of idiots...why would they do this...now what for my workflow?

    Since I'm that guy, I'll explain what I was thinking. We have millions of customers and they have customers and speed is crucial to probably all of them. And with this change we could get nice speed gains for most customers. And, sadly, we have less than 1,000 customers who've looked at their stats since we made this change. Maybe we bury them too deeply, or they're not that important to most people, I don't know. But in the land of tradeoffs, I was doing the math between making millions happy and hundreds sad (although I didn't know how sad at the time).

    Now that I just pissed some of those hundreds off with the previous paragraph, what's to do about your workflows? We're looking into the possibility of excluding owner views with our new delivery network. I honestly don't know how likely it is that we'll succeed or how long it would take, only that we're working on it and serious about it.

    Some of you might ask, why would we be serious about it? Millions are happy, a few hundred are mad... Is Baldy just saying that to appease us?

    And the answer is we invest over a million dollars a year collecting, storing and displaying stats. It's really a shame to make them useless to a significant number of the people who really care about them, many of whom are among our best customers. Although I often wonder if we can't simplify stats like other sites do to make the problem easier (and stats more popular).

    I hope this helps.

    Thanks,
    Baldy
Sign In or Register to comment.