29. Help, Book Library does not fetch my book descriptions!

In this article we will take a closer look at why the fetching of book descriptions from Amazon only works consistently from the US site, and inconsistently from other Amazon sites. It all has to do with copyright restriction at Amazon, not with Book Library programming...

The problem:

Book descriptions are only returned consistently from Amazon.COM.DE, FR, UK, JP and CA will not return book descriptions to queries many times. So these Amazon sites return book descriptions every now and then, without consistency.

The answer:

We write in the Book Library Shop features overview that the book description fetching only works reliably from Amazon.com. And we do that for a good reason, we almost went bonkers trying to figure out why it did not work for the other Amazon sites. So to summarize the rules for which descriptions are available through Amazon Web Services:

  1. Only Editorial Reviews that are listed on amazon.com as [Editorial Review] are available via AWS.
  2. It is not unusual, especially in non-US locales, for this data to not be available. This is often due to legal or contractual limitations imposed upon Amazon by the data provider.
  3. Amazon can display editorial reviews on their own HTML pages, because they have received permission from the appropriate copyright holders to publish these reviews. The permission to publish copyrighted material does not include permission to serve as a republisher of the same material (i.e., provide the data via AWS)

More explanation:

The Amazon Web Services developer blog is full of questions (since 2002/2003…) pertaining to why the book description is not returned on Amazon DE, FR, UK etc. Just go to http://developer.amazonwebservices.com/ and search for editorial, review, editorial review, book description, description, review. I have read through them all, but I will give you the lowdown based on the few proper responses from Amazon staff. Quotes from the horse’s blog:
  1. The lowdown from Amazon
  2. The problem descriptions
  3. Copyright issues
  4. Technical specification Editorial Reviews
  5. REST issues
  6. Possible Amazon server issues

The lowdown from Amazon

  1. Just to clarify: only Editorial Reviews that are listed on amazon.com as [Editorial Review] should be available via AWS. We currently researching issue why some reviews even though written by Amazon.com are not available via AWS. When we fix the issue , reviews that are not written by Amazon.com will still NOT be available via AWS at present.
  2. When the data is available it is usually under Editorial Review. It is not unusual, especially in non-US locales, for this data to not be available. I believe this is often due to legal or contractual limitations imposed upon Amazon by the data provider. Legal restrictions prevent us from providing all editorial reviews. This is mentioned in section 3.2 of API and Integration Guide and repeated below.
  3. Amazon attempts to return as much data as possible for each query. In those cases where Amazon is not the primary owner of the data (such as third-party reviews), we may not be legally able to release that data. Our goal, however, is to return as much product information as we can.
  4. Currently, any review written by Amazon.co.uk (or .com) should be available throughAWS (note that this does not necessarily include all reviews that you may see on the retail site). If you see a discrepancy between AWS responses and the retail website, this may be a bug. Posting specific examples will help in determining this
  5. Amazon can display editorial reviews on their HTML pages, because (presumably) Amazon has received permission from the appropriate copyright holders to publish these reviews. The permission to publish copyrighted material does not include permission to serve as a republisher of the same material (i.e., provide the data via the AWS feed to all comers). Note that since Amazon.com holds the copyright for all of their HTML pages, (see the Copyright Section of http://www.amazon.com/exec/obidos/tg/browse/-/508088/102-6202356-3181709 ), the HTML pages do not serve as a republishing source.
  6. Editorial Reviews are only available via AWS when written by Amazonians. Good way to check --look at the website: http://www.amazon.com/exec/obidos/ASIN/0399238611. If you see: Editorial Reviews Amazon.com --it is a good indication that review should be available via AWS. Problem descriptions:
  7. I'm suffering the same problem but the whole copyright issue seems a bit contradictory: Take the book "The Acne Cure" for example (ASIN: 0446692417):-Editorial Reviews show up in my UK a store ( http://astore.amazon.co.uk/megavista-21/)-Editorial Reviews show up on amazon.com (many)-Editorial Reviews shows up on amazon.co.uk (few)-Editorial Reviews are AVAILABLE in .com XML-Editorial Review are missing from .co.uk XML That seems strange to me if this is supposed to be a copyright issue. How can US associates be allowed to display review data on external sites without breaching this copyright? (no reviews for this book mention they are by Amazon). The .com XML only contains the book description and not all the reviews on the .com site but this is better than nothing.
  8. I'm using the UK version of AWS and no editorial reviews or features are being returned, despite using the "EditorialReview" and "ItemAttributes" response groups.
  9. OK, I'm getting the impression from this and other threads; UK coverage in this area is patchy at best, i.e. not to be relied on. Well, I'm sorry to say this has been the situation for years. I think we can assume it will be the situation for years more.
  10. We are trying to sell Amazon products -and develop systems to allow others to sell Amazon products -but Amazon doesn't seem too bothered about giving us the tools to do so. AWS has a lot of potential -but I believe Amazon need to commit a little bit more effort in order to make it really live up to that potential. I appreciate that the Amazon staff in here probably can't do anything until we show them something that doesn't work as 'expected'. The problem is that they (Amazon) and Us (associates) have different expectations. If it works according to spec -to Amazon, it isn't broken. To us the associates -it is not the implementation that is broken, the spec itself is broken -it doesn't conform to our expectations. Too few products have usable Amazon.co.uk reviews. I re-iterate my opinion that we could do with greater coverage of Amazon.co.uk reviews.
  11. Is it possible to receive either the Publisher or Amazon.com Editorial reviews via AWS?
  12. I get product description but not consistently . Most products are listed without product description. Also please let me know if no product description is available then can I get editor review?

    13. Copyright issues:

  13. Most "Book Descriptions" are supplied by the publisher of the book, and therefore Amazon.com does not have permission to use it in our XML documents. The same is true for our third party editorial reviews.
  14. The "Book Description" is from the publisher and is copyrighted. It is sometimes called "From the Inside Flap" or "Synopsis". You won't find it in AWS--only Reviews written and owned by Amazon are available in AWS.
  15. You can get product descriptions written by Amazon.com via AWS. However, AWS will not return product descriptions or editorial reviews written by third parties, such as Publishers Weekly or Library Journal due to copyright issues. This means that the Product Description element will only be returned for items for which Amazon has provided an editorial review or description.
  16. Is it not time--and past time--that those amorphous "issues" got resolved? The reviews appear on Amazon's own site, so _some_ permissions, or some application of the "fair use" doctrine, are already in place. Whatever forbids associates, who are effectively an extension of Amazon's site, from using that material? Amazon's lawyers should look at this, and settle it that the material--often quite important (for example, often the only way to find out what's in a story collection or an omnibus volume) *is* usable. (My strong suspicion is that no one has ever sat down and said, "Look, you can't do this"--rather, that no one has ever sat down to actually decide, so everyone's just playing it safe.) The failure to take the minimal time and effort needed clarify the ability of Amazon associates to display material that Amazon can and does display is just another sign of how high associate concerns do (or don't) rank within Amazon.
  17. Product descriptions and reviews are different things. In this thread, we are discussing product descriptions, which are general synopses or descriptions of products written either by Amazon or by third party content providers Amazon has formal partnerships with. Reviews are what Amazon customers write about products. Through AWS, you can retrieve the first three customer-written reviews for any product. At the present time, it is not possible to retrieve all the reviews written by a specific customer. It is only possible to retrieve reviews by product.
  18. Amazon does not own all the editorial content that appears on the retail site and some of it is licensed from third parties who limit the ways in which their intellectual property is allowed to be reproduced. As a result, we need to filter out some editorial reviews from public responses.
  19. Amazon only had the right to use descriptions like these themselves and not to let others (like associates as us) use them. I assume that on a per seller basis this is being administered and that items for which this isn't cleared yet, Amazon doesn't provide this information yet.
  20. It is in the Editorial Review response group when it exists. Here is an example. The Source value will be your cue. Product DescriptionA bite-sized version of our famous Victoria Toffee is ideal to enjoyanywhere. Almost everyone loves this crisp, smooth, crunchy combinationof butter brickle, milk chocolate and almonds. Net Wt. 1 lb. CertifiedKosher

    21. REST issues

  21. It seems to me that the problem has been and still is that Product Description is NOT being returned via REST/XML.
  22. The answer you provided to the previous post on this issue does not answer the question of why the Product Description would not be returned for service (REST). If the case were just that "AWS will not return product descriptions or editorial reviews written by third parties, such as Publishers Weekly or Library Journal due to copyright issues" -then obviously the Product Description would not be available via either SOAP or REST. It seems to be a tech/programming issue and not a copyright one.
  23. I've been experiencing the same problem with the 'ProductDescription'. I was trying to find books in the category '1201358' via the 'BrowseNodeSearch'. However, of the 190 books returned, only 38 have a product description. The books were retrieved with locale=de (German). This is the same no matter how I retrieve them -with REST/XML or SOAP (schema3 used) -even when trying an 'ASINSearch' the description is still not shown.
  24. I have found that getting the EditorialReview for toy items is non-existent. I have tried several things. When I run my code against a DVD product, I get the data. Can someone explain why? The toy products on the Amazon site (US) indeed have EditorialReviews. In my opinion, this is one of the most important pieces of data, as most people refer to this as the "description" of what they are buying.
  25. It looks to me like the response from A2S is in line with the data available on the amazon.fr site. You'll find additional customer reviews by including the Reviews response group in your A2S request.
  26. I've experienced, and read multiple complaints about, issues where there is simply no Editorial Reviews available in the SOAP Responses when they are clearly available on the website in the UK locale. Based on other complaints, this is obviously wide-spread across other locales and everyone has the same exact complaint: that the Editorial Reviews are available when using US locale, but not the others. And every single Amazon rep who has answered this has given some lame response that "they" are seeing it fine. Is that in any way a professional response. This is quite obviously a very REAL issue and every time it's brought up, an Amazon rep acts like there is nothing wrong. Are you kidding me? Use the service like anyone else has and you'll see... there are definitely issues with the Editorial Reviews for locales other than US. I know it's so hard to believe guys, but trust the numbers... WE ARE ALL HAVING THIS ISSUE!!! IT'S REAL!!! Please fix it, please. Pretty please. People have been complaining about this issue for years and I'm shocked there is nothing but open-ended questions on these boards.

    27. Possible Amazon server issues

  27. Amazon uses a huge number of servers behind one DNS name. Sometimes information that is available through SOAP or REST will be returned to the query, sometimes it will not be returned.
  28. Database consistency not the same over all servers?
  29. Server too busy does not return all results?
  30. I have not seen a comment on this problem. Could I at least get a confirmation that there is something wrong on that end and I am not doing something incorrectly? Thanks. The problem of missing Editorial Reviews is still consistent. In some instances I get only one editorial review when on Amazon there are multiple editorial reviews. But for the most part, they are missing all together. Someone of AMZ technical ability and insight please comment on the issue.
  31. Amazon, August 2006: fix for editorial reviews is out, and you should be able to see Amazon.com editorial reviews as well as product descriptions via AWS. Reminder: To check whether particular editorial review should appear in AWS, please look at the source of the review on the website. If source is either "Amazon.com" or "Product Description" / "Book Description", review most likely should appear in AWS (there are some exclusions from this rule -we'll need to check individually if you come across such review). So there you have it. This is a general issue that everyone is experiencing with Amazon, through SOAP and REST requests, through BookLibrary or another tool. It has too do with Amazon policies, not with BookLibrary coding errors. Sure, we could parse html web pages and grab the results, but that would again be a copyright infringement.

    32. Conclusion:

  32. Book descriptions can be pulled in from Amazon.com (US) consistently.
  33. For ALL other Amazon sites, we cannot guarantee anything. If you get some descriptions, you're already lucky. This means you will have to manually copy & paste for those descriptions in other languages, but be aware of possible copyright infringements! We do not take responsibility for that, that responsibility is your own only!
  34. Amazon decides which reviews and descriptions to make available through their Web Services, not Book Library...

The good and the bad

Let's not forget that Amazon is providing us with a FREE service, by allowing access to their databases. Of course they want to sell products through affiliates, but they still allow ANYONE with a developer's key access through SOAP or REST to their databases, and they have a wonderful, very advanced API for that.

Please try to remember that it is already fantastic to be able to pull in as much data as we do automatically, it really beats doing everything by hand! Also, Amazon is one of the VERY FEW book sites which have an API out for this purpose at all. Most other book stores force their affiliates to only create links in very cumbersome ways in the exact way they specify. So let's count our blessings, shall we?

View Demo Version

or

Go to Full Description
What are you looking for?
»