ŷ

ŷ Librarians Group discussion

154 views
Questions (not edit requests) > Export file is missing data -- how to fill the holes?

Comments Showing 1-24 of 24 (24 new)    post a comment »
dateUp arrow    newest »

message 1: by Mesembryanthemum (last edited Jul 13, 2023 01:06AM) (new)

Mesembryanthemum | 191 comments Like many others, I have noticed that the GR bots are doing a lot of damage to existing book data. (For details, see the thread Amazon destroyed the author's database.) That's why I export my book data often.

But GR's export file is missing lots of information. For example:
- No read date for anything I’ve read in 2022 and 2023
- Missing multiple read dates for ANYTHING I’ve read, ever (and I’ve read many, many books several times)
- Bad links to images in my reviews (the exported link is obviously internal to GR; it’s not the same link I see when I view my review)
- No "sort-by" title
- No ASIN for Kindle editions (or print editions, but who cares about those)
- No edition language
- No edition details (the contents of the “edition� field is missing)
- No book description
- No series information (I want to see each book’s “number� in the series)
- No other work-level details, other than the original publication date

Any tips for how to retrieve the missing information � which is clearly available when I view my shelved books on ŷ � and add this information to my export files?

Please note that I export my data at least once a week, if not daily, and I have almost 4000 books on my shelves. So I can’t spend much time on correcting each export file by hand.

Also, please note that I’m not a programmer, but I know how to use Linux tools like vim, ed, sed, awk, and the like. In fact, I can even write Linux shell scripts. (Whoop de doo!) So I’m willing to spend a little time on automation if it will lead to correct and complete export files.

P.S. No, I haven't reported these problems to support. They haven't fixed any of the bugs I've reported before. Plus, several of the above problems have been known -- and reported, over and over -- for years. (I searched for "export" on help.goodreads.com before I posted this, and many users have complained about problems with the export file, to no avail.) So it doesn't seem like a good use of my remaining time on this planet. Instead, I want to spend my time extracting accurate, complete data for the books I've shelved on ŷ.


message 2: by Scott (new)

Scott | 4605 comments Glad to know the export works as well as anything else here.


message 3: by Elizabeth (Alaska) (last edited Jul 13, 2023 06:29AM) (new)

Elizabeth (Alaska) Mesembryanthemum wrote: "- No read date for anything I’ve read in 2022 and 2023"

The file I exported earlier this month has those dates.

Many of the other things you report as missing have *never* been in the export file: edition, language, description, series info. Please note that those things aren't information on your shelves. The things in the export (except for ASIN) are the things that you could display on your shelves. It is a shelf export, not an export of all GR data.

I wouldn't even *want* much of the data you complain about not having


message 4: by Emily (new)

Emily | 17067 comments The main thing that should be on there that isn't is the read dates. The single read sessions at least should be. If you ever feel like making another report, that is the data I would emphasize. Unfortunately I have no idea of a fix.


Elizabeth (Alaska) Emily wrote: "The main thing that should be on there that isn't is the read dates. The single read sessions at least should be. If you ever feel like making another report, that is the data I would emphasize. Un..."

The Read dates *are* there. There is no report to make.


message 6: by Elizabeth (Alaska) (last edited Jul 13, 2023 08:06AM) (new)

Elizabeth (Alaska) Corinne wrote: "You make a good point Elizabeth but there are some things in the export file that aren't on our shelves. i.e. publisher, additional authors. Which I found handy on more than one occasion.
"


I think the export data file was so that we would always have what is on our shelves. Additional authors, for example isn't on our shelves. (ETA: But it *is* on the export file.)

Publisher is in the export file. It is in the column between average rating and binding.

EDIT: Below are the columns on the export file. I note that Additional authors is included, which I had not ever thought to look at before. I don't know if that column includes *all* of the additional authors on a record.

Book Id
Title
Author
Author l-f
Additional Authors
ISBN
ISBN13
My Rating
Average Rating
Publisher
Binding
Number of Pages
Year Published
Original Publication Year
Date Read
Date Added
Bookshelves
Bookshelves with positions
Exclusive Shelf
My Review
Spoiler
Private Notes
Read Count
Owned Copies


Elizabeth (Alaska) Corinne wrote: "I understand that but the ability to add information not on our "shelves" is there and can be beneficial for those interested in the information especially since they are not searchable on our shel..."

Ok, we have different opinions. Further, this entire thread isn't a subject for the librarian group.


Elizabeth (Alaska) Mesembryanthemum wrote: "No series information (I want to see each book’s “number� in the series)"

The series information is there is the series is in the title. If it isn't in the title, you couldn't see it on your shelves.


message 9: by Emily (last edited Jul 13, 2023 08:37AM) (new)

Emily | 17067 comments I meant that if the read dates are on the books, then they should be on the exported file. I exported mine to check and the ones that I have completed are on my report. So if they aren't on the OP's then there is a glitch.


message 10: by Elizabeth (Alaska) (last edited Jul 13, 2023 08:43AM) (new)

Elizabeth (Alaska) Emily wrote: "I meant that if the read dates are on the books, then they should be on the exported file. I exported mine to check and the ones that I have completed are on my report. So if they aren't on the OP'..."

As I said, this isn't a report for the librarian group. If the OP has a problem with an export file, then it should be reported to support. I'll just say that *all* of the information that should be exported is in my file (and that is more information than I need or want).


message 11: by Emily (last edited Jul 13, 2023 08:53AM) (new)

Emily | 17067 comments I was clarifying my statement earlier. There are plenty of other threads that are about site functionality or with people insulting librarians that should be closed anyway, so because of that some people may not realize that a question like this doesn't belong. Now they will know for next time.


message 12: by Shaz (new)

Shaz | 21 comments Elizabeth (Alaska) wrote: "The Read dates *are* there. There is no report to make."

The column is there, unfortunately a lot of information is missing! I exported my data at the end of last month. Of my 3580 or to books read over 800 read dates were missing. And trust me, they are all listed on GR. Funny thing was though, they were all from 2017 onward, with over 400 titles from 2021, 2022 and 2023.

So yes, there is a case for a report to be made. And while the last read date is often available (although a few of my 800 missing dates had been read more than once) the earlier read dates aren't exported either. So that's another job I need to go back to and add manually to at least get a spreadsheet that is correct.

And while I understand this isn't really something the Librarian's Group can or needs to deal with, there really isn't anywhere else to discuss it in reality. In the Help section? You either get an extremely unhelpful comment from a volunteer who can't do anything about anything telling you to contact support, or the thread gets closed down because the PTB don't want to deal with it...


message 13: by Mesembryanthemum (last edited Jul 13, 2023 10:08PM) (new)

Mesembryanthemum | 191 comments Thanks, Corinne, for the history of the export function. It seems that poor little export has been languishing without support for a long time.

For me, being able to export complete information is a librarian issue. My goal for exporting data is to protect book records from bot damage � for the books I’ve shelved, at least � to the extent of my ability.

Yes, of course I want to save my personal data, including my multiple read dates. But I also want a complete record for the books I’ve shelved, so that I can use it to repair problems caused by the MadBots, if (when) they mess it up, without leaving enough information in the change log. Alas, this is something that seems to happen more often these days. The missing stuff in the export file makes the repair task more difficult.

But I'm not looking for changes from GR developers. It's very clear in the Help section that export is not a priority for the staff. I'm just looking for tips and advice on capturing more information so I can fix MadBot problems.

Edited to fix the verb tenses in my overly complex sentences. Sigh ...


message 14: by Mesembryanthemum (last edited Jul 13, 2023 10:05PM) (new)

Mesembryanthemum | 191 comments Elizabeth (Alaska) wrote: "The Read dates *are* there. There is no report to make."

Shaz wrote: "The column is there, unfortunately a lot of information is missing! I exported my data at the end of last month. Of my 3580 or to books read over 800 read dates were missing.


I'm glad to know that I'm not the only one who has seen this problem. (Not really -- I'm sad about this, because it means that it wasn't just me leaning on the keyboard.)

Hmmm... I wonder if this is a glitch that happens when the GR server is stressed. The different years made me wonder (me: 2022-2023; Shaz: 2017-on). So I re-checked all my export files, and the problem seems to come and go. In some files (like one from two months ago), all my read dates seem to be there. In others, like this week's file, all the recent dates were missing. The number of affected read dates seems to change from file to file.

So now I want to know what else might be dropped when the server isn't happy. Maybe the ASINs really are there -- but only when the server has a 5% load or less. (Just joking...)

Anyway, my new motto is "Export often and double-check everything." And also "I need a better spreadsheet comparison tool."


Elizabeth (Alaska) Shaz wrote: "And while I understand this isn't really something the Librarian's Group can or needs to deal with, there really isn't anywhere else to discuss it in reality."

It isn't something for librarians to discuss. You must report to staff. Period. Further, I looked at my export file and all (yes, all) of the read dates are included.


message 16: by Mesembryanthemum (new)

Mesembryanthemum | 191 comments Thanks for the explanation.

Message understood. Over and out.


Carol She's So Novel꧁꧂  | 1958 comments Mesembryanthemum wrote: "Thanks for the explanation.

Message understood. Over and out."


Maybe you could form a group?

I have problems with my export too & I would certainly join!


message 18: by Dobby (new)

Dobby (dobby0390) | 7810 comments I would join your group, too.


message 19: by Cell (new)

Cell | 3 comments Regarding blank read date in the export file

Possible cause:
The books were directly marked as read (i.e. the "currently reading" button was never pressed)

How to find the books with read-date holes:
1. Locate them in the export files
2. Open the GR app and go to "My Books" →sort by date finished or date started

How to fill the holes:
Go to "My Books" page in PC
Find the books with holes (although they look fine in the page)
Click "edit" in the date read column
Click "save"


Carol She's So Novel꧁꧂  | 1958 comments Cell wrote: "Regarding blank read date in the export file

Possible cause:
The books were directly marked as read (i.e. the "currently reading" button was never pressed)

How to find the books with read-date ho..."


Thank you!


message 21: by Mesembryanthemum (new)

Mesembryanthemum | 191 comments Carol, Dobby, Shaz, and all others who are interested, I cordially invite you to join the new "Export" group to talk about the export feature.

Just in case it's against the rules, I won't post the group's link here. But you can check my profile (I'm only in two groups). Or you can search for "export" on the main Groups page.

See you there, I hope!


message 22: by Dobby (new)

Dobby (dobby0390) | 7810 comments Thanks for setting this up, Mesembryanthemum.


Carol She's So Novel꧁꧂  | 1958 comments Mesembryanthemum wrote: "Carol, Dobby, Shaz, and all others who are interested, I cordially invite you to join the new "Export" group to talk about the export feature.

Just in case it's against the rules, I won't post th..."


Thank you! See you tomorrow!


message 24: by lethe (new)

lethe | 16354 comments Thanks, Mesembryanthemum.


back to top