Bug: Keyword indexing not working - Affects smart galleries etc

edshedsh Registered Users Posts: 27 Big grins
edited October 7, 2013 in Bug Reporting
Hi,

Seems keyword indexing dont work and the effect is that Smart Galleries with excluding these keywords dont work.

Use case:
1. Added keywords to images.
2. Created a smart gallery based on both include AND exclude of keyword rules (match rules: ALL)
3. Refreshed smart gallery
4. Result: It does NOT exclude images with the excluded keywords

Analysis:
* When editing caption & keywords: Images have the keywords on them
* When checking keyword page, the keyword is not found
* Sometimes, keywords are saved with ; instead och , as separator

Conclusion:
The keywords are not indexed, even as they are on images. Therefore they are not excluded from my smart gallery.

Questions to SM:
1. Any existing bugs for keyword indexing ?
2. Any possibility to see indexing-status (on legacy SM we could see it by "indexing: <keyword>") ?
3. How long time does it take to index a keyword?

Comments

  • rsigginsrsiggins Registered Users Posts: 4 Beginner grinner
    edited August 11, 2013
    I'm seeing the same behavior. I noticed it before I upgraded my site and attributed it to server overload. Now, I'm not sure that's the case. I'm not finding a lot of other posts about this problem so it may be new.
    edsh wrote: »
    Hi,

    Seems keyword indexing dont work and the effect is that Smart Galleries with excluding these keywords dont work.

    Use case:
    1. Added keywords to images.
    2. Created a smart gallery based on both include AND exclude of keyword rules (match rules: ALL)
    3. Refreshed smart gallery
    4. Result: It does NOT exclude images with the excluded keywords

    Analysis:
    * When editing caption & keywords: Images have the keywords on them
    * When checking keyword page, the keyword is not found
    * Sometimes, keywords are saved with ; instead och , as separator

    Conclusion:
    The keywords are not indexed, even as they are on images. Therefore they are not excluded from my smart gallery.

    Questions to SM:
    1. Any existing bugs for keyword indexing ?
    2. Any possibility to see indexing-status (on legacy SM we could see it by "indexing: <keyword>") ?
    3. How long time does it take to index a keyword?
  • edshedsh Registered Users Posts: 27 Big grins
    edited August 11, 2013
    Tried to give it a bunch of hours to see if keywords getting indexed. And they're not (yet anyway).

    This is quite a nasty problem for me because my whole structure of galleries depends on smart galleries based on keywords.

    Smugmug? Any comment on this?
  • mbonocorembonocore Registered Users Posts: 2,299 Major grins
    edited August 11, 2013
    We are plowing through keyword indexing, but yes, due to the heavy influx of new photos added since launch, they were backlogged. Keep checking, they will be there. Sorry!
  • edshedsh Registered Users Posts: 27 Big grins
    edited August 12, 2013
    mbonocore wrote: »
    We are plowing through keyword indexing, but yes, due to the heavy influx of new photos added since launch, they were backlogged. Keep checking, they will be there. Sorry!

    Oki thanks, great to hear that it's under the loupe :)
  • GloriousnseGloriousnse Registered Users Posts: 29 Big grins
    edited August 12, 2013
    mbonocore wrote: »
    We are plowing through keyword indexing, but yes, due to the heavy influx of new photos added since launch, they were backlogged. Keep checking, they will be there. Sorry!

    Sorry to be a little pushy, i'm broadly very happy with the new Smugmug, but this is affecting something that is 'core functionality' for me - do you have a guide figure for how long it *should* be taking under the current workload? I don't recall simple backlogs taking this long to process through before, and i'd hoped things would have been gradually improving now we're nearly 2 weeks into The New Hotness? mwink.gif

    At the time of writing i'm now at 33+ hours waiting for (most) of my images uploaded on Saturday to be properly indexed. (Just as a test, I've just searched on the 'date' tag (10082013) I gave all that days images, and it comes up with only 26 results (apparently at random) out of over 150 images uploaded)

    If you're going to tell me it's currently taking multiple days to do this at present that's fine, i'll leave it a couple of days before checking back, i'm just a little worried there is something more than load going on, as i'd suffered from 'stuck' galleries less than a month before on old Smugmug.
  • kimbomackimbomac Registered Users Posts: 99 Big grins
    edited August 12, 2013
    I am also unable to properly structure my new site without the smart galleries working properly. The search functionality is also not finding all the images with a given keyword.

    This makes the search a laughable function, and ditto the status of my smart galleries.


    Please can you let us know what the problem is, and when it might be resolved.
    Kimbomac

    (FeaturePhotography.net)
  • phaserbeamphaserbeam Registered Users Posts: 452 Major grins
    edited August 12, 2013
    Since indexing newly uploaded photos is taking longer then expected (have a look here) i believe this is just another problem based on the heavy load for the indexer.

    If there is really no other problem, then i have currently 3 photos in 6 hours being progressed. So indexing keywords should take as much time...
  • GloriousnseGloriousnse Registered Users Posts: 29 Big grins
    edited August 12, 2013
    So now i'm over 45 hours from upload, I did a quick count up and it was 238 images uploaded that evening - I've just run some quick tests.

    * Number that come up with a search on a keyword they all had (as part of the filename) - 46 of 238

    * Number that show up on 'Latest' - 45 of 238

    * Number that appear to have indexed properly to (any of!) the galleries they were supposed to - 0 of 238

    *I reset the smart gallery terms on some galleries that should have seen images from those 46, refreshed, Number that have indexed properly - still 0 of 238

    *Created a temp test gallery, gave it the smart term as per the search in step 1 - smart gallery has 0 images in it

    *Created second temp test gallery, gave it the date 10th Aug as a smart term - all 238 images found

    Sorry guys, I think it's broken not slow, your system knows those images exist, but a keyword search can't find most of them. Even on the ones where you can search and your system has indexed and can find the keyword in a search, the smart gallery system still will not find them...
  • phaserbeamphaserbeam Registered Users Posts: 452 Major grins
    edited August 12, 2013
    Sorry guys, I think it's broken not slow, your system knows those images exist, but a keyword search can't find most of them.

    Well... i changed a keyword in about 20 photos and right after i did that the search did not find any of these images. This was about 24h ago.
    Now the search finds about half of the photos with the replaced keyword. So it really looks like the indexer is working, it's just horrible slow.

    I wonder if that is going to be normal for now and in the future or if the indexer is someday going back to be faster even when people continue to upload new photos.

    On weekend there will be great fireworks for about three days. That will be 200-300 photos. With a indexer being slow like that the recent photos block or the keyword search will take weeks or months to catch up. :cry
  • mbonocorembonocore Registered Users Posts: 2,299 Major grins
    edited August 12, 2013
    This will not be the new norm. The keywords are catching up faster and faster and we have made adjustments to handle indexing much faster.
  • phaserbeamphaserbeam Registered Users Posts: 452 Major grins
    edited August 13, 2013
    mbonocore wrote: »
    This will not be the new norm. The keywords are catching up faster and faster and we have made adjustments to handle indexing much faster.

    Thanks for the reply... so we just need to wait a bit longer. rolleyes1.gif
  • smtownssmtowns Registered Users Posts: 1 Beginner grinner
    edited August 13, 2013
    I am having a similar problem with 2 new photos uploaded after I migrated. The whole site went over cleanly so was surprised when 2 photos are still broken. I am only using include keywords in my smart gallery. My whole site is dependent on smart galleries too.... very frustrating.

    Will also check back in another day.... since i have family waiting on me now. :-(
  • dennismullendennismullen Registered Users Posts: 709 Major grins
    edited August 13, 2013
    Days have gone by without indexing!
    See my gallery at http://www.dennismullen.com
  • GarryGarry Registered Users Posts: 8 Beginner grinner
    edited August 15, 2013
    Smugmug team, please let us know you are working on this.
    I have

    1. Added keywords to images.
    2. Created a smart gallery based on Keywords
    3. Refreshed smart gallery
    4. Result: It does NOT fetch images with the keyword(s)

    Analysis:
    * When viewing images (logged out): Images have the keywords on them
    * Did another test to with a different rule base on date. RESULT: It fetched the images correctly
    * The keywords are NOT listed in the keywords list
    * When changing the rule from Keywords to Gallery the Gallery will not show up
  • dennismullendennismullen Registered Users Posts: 709 Major grins
    edited August 15, 2013
    Still no keyword indexing after many days!
    See my gallery at http://www.dennismullen.com
  • GaryBakkerGaryBakker Registered Users Posts: 266 Major grins
    edited August 15, 2013
    A few days ago I did a test with a single keyword and a single smart gallery, to see how long it would take. It has been four days and the gallery is still empty.
    SmugMug site => The Bakker Chautauqua
    "The difference between stupidity and genius is that genius has its limits." (Einstein)
  • OjaOja Registered Users Posts: 76 Big grins
    edited August 15, 2013
    Yeah this is starting to look more like bugs than an indexing problem. I have images that show up on a specific keyword search but that are not placed in the smart galleries. An example is http://lagemaatphoto.smugmug.com/keyword/mountain The second and third image clearly have keywords mountain and colorado and are found in direct keyword search, yet they don't show up in a smart gallery that searches for "colorado" and "mountain" (http://lagemaatphoto.smugmug.com/Landscapes/Colorado-Rocky-Mountains/All-Colorado-Mountain-images ) . These were uploaded 4 days ago. The third and fourth images in http://lagemaatphoto.smugmug.com/keyword/mountain were uploaded the day of the new smugmug release as my unveil killed a lot of images on my site that support couldn't find anymore (a combination of the smugmug lightroom plugin misbehaving and weirdness with smart galleries in the organizer tool). They should also be in http://lagemaatphoto.smugmug.com/Landscapes/Colorado-Rocky-Mountains/All-Colorado-Mountain-images but are not. They do show up in random other smart galleries where they should such as http://lagemaatphoto.smugmug.com/Landscapes/National-Parks/Rocky-Mountain-National-Park/Tyndall-Gorge . Refresh doesn't help on any of these.
  • joshwiselyjoshwisely Registered Users Posts: 5 Beginner grinner
    edited October 6, 2013
    The indexer has been very slow since the big update. Shortly after the update it was about a week behind in processing updates. Since then it has improved to as little as an hour, but often seems to take many hours to process updates.

    I've asked Support to list the current lag time on the status page so I could at least understand how far behind the indexer was, but all I hear is crickets on that request.

    The indexer also seems to have a more serious problem in that is sometimes fails to index photos properly even after it runs against a photo. I've started a new thread under bugs for this issue. http://www.dgrin.com/showthread.php?p=1916687#post1916687
  • JHDesignJHDesign Registered Users Posts: 13 Big grins
    edited October 7, 2013
    Same Problem? Trouble with Shared PP Smart Galleries
    After losing hours of my life trying to figure out why something I have done frequently is no longer working as expected, I finally am searching for answers here.

    To share photos with friends, or with clients, I often tag the images I want them to see with an index keyword, then create a smart gallery based on that keyword, then create a share group containing that smart gallery. Since I want the photos to be pw protected and unlisted, I assign a pw to the smart gallery and then assign this same pw to the sharegroup pw access. Then I share the url and the pw with whomever and they use that to access the photos I want them to see. It's been a snap! I've had no problems with this and it has been a great way to only flag some of my protected photos for viewing by third parties.

    I tried doing this same set up (starting on Sat.) and it didn't work. "Didn't work" looks like this: the smart gallery creates beautifully (to me as a signed in SM user) and no problems setting up the share group. I test by signing in with the sharegroup url from another browser. I can sign in with the password and am clearly "in" the gallery but there are only placeholders for the photos and no photos can be seen. (Looks like an box with a red "x" on it in IE and a broken document icon in Google.) Everything appears normal--layout, number and orientation of photos, descriptions, etc.--except I cannot see the photos.

    Why can I not see the photos? I can still access and see other pw-protected/share grouped smart galleries I made in the past (the most recent one was at the end of the summer) but I can no longer, it seems, do what I have done before. (I'm still in legacy mode as I am loathe to switch over due to being busy with work and not wanting to take time away from work to figure out the new Smugmug.)
  • JHDesignJHDesign Registered Users Posts: 13 Big grins
    edited October 7, 2013
    GaryBakker wrote: »
    A few days ago I did a test with a single keyword and a single smart gallery, to see how long it would take. It has been four days and the gallery is still empty.

    I'm trying to figure out if this is the same issue I've been having. Does "empty" mean even while signed in you cannot see the photos in the smart gallery you created? Or is it only appear empty if you share that link with a non-smugmug account holder? I'm asking because it appears that I'm able to use keywords and create smart galleries as before--everything pulls in beautifully. But the photos aren't really "there" when I try to share this link with someone else (meaning they can sign in or access the gallery but there are only placeholders there instead of photos.)
  • JHDesignJHDesign Registered Users Posts: 13 Big grins
    edited October 7, 2013
    Just an addendum to my previous posting--the keyword indexing issue may NOT be my problem because the keyword in question is available from the list of keywords for my site. (So is listed there and I can choose it and the photos also pull into/populate the preview window just fine.) Everything is fine except the photos showing up to a third party.
  • JHDesignJHDesign Registered Users Posts: 13 Big grins
    edited October 7, 2013
    JHDesign wrote: »
    Just an addendum to my previous posting--the keyword indexing issue may NOT be my problem because the keyword in question is available from the list of keywords for my site. (So is listed there and I can choose it and the photos also pull into/populate the preview window just fine.) Everything is fine except the photos showing up to a third party.

    Hours later and my smart gallery still is hiding images. (Not to me as a signed-on user, but I can't give out a link to images because the gallery loads but the images are only place-holders.) I posted this problem again as a separate thread, with more detail.

    My biggest discovery was that my hidden image problem was only occurring with smart gallery images. (I have two layers of pw lock-down so worried at first there was something wrong with my permissions' settings. But I'm able to set up everything as expected as long as a smart gallery isn't in the loop. The share group part of the equation also doesn't seem to figure in since I can create share groups, just not including smart galleries since those galleries will allow access but will not show the images.) Internally when creating a smart gallery it appears the images have indexed (the term shows up in the keyword list and the preview populates the smart gallery and it also appears as a populated gallery on my homepage). But something is wrong because the gallery is still not populated with images from the link I send out.

    I finally gave up and just uploaded the images I needed to provide a link to, again, to their own separate pw protected gallery and just gave the link to that. I obviously don't think it's good practice to double load everything into separate galleries but I need to move on and this allows me to do so! I'm washing my hands of this for today.
Sign In or Register to comment.