| Next revision | Previous revision |
| oldtech:calibre:decrapifyandsend [2025/05/07 16:49] – created ifixcoinops | oldtech:calibre:decrapifyandsend [2025/05/15 19:04] (current) – ifixcoinops |
|---|
| ===== Finally time to plug in the ereader ===== | .====== Decrapify and Send ====== |
| | |
| | It's finally time to plug the ereader into the computer! |
| |
| When you first plug in the ereader, Windows will make a bunch of noises. It might complain that there's a problem with the drive you just plugged in - ignore it, it's lying, the drive is fine, do not accept its offer of help. Wait for Windows to calm down. Once it's stopped pinging at you, Calibre should have added another couple of buttons to its top toolbar; next to "Library," sixth from the left, will be "Device" and "Card A." If you click on either of those two buttons, it'll show the books in the main memory or SD card of the ereader. Have a look if you like, there might be stuff in there already or there might not, and then go back to Library. | When you first plug in the ereader, Windows will make a bunch of noises. It might complain that there's a problem with the drive you just plugged in - ignore it, it's lying, the drive is fine, do not accept its offer of help. Wait for Windows to calm down. Once it's stopped pinging at you, Calibre should have added another couple of buttons to its top toolbar; next to "Library," sixth from the left, will be "Device" and "Card A." If you click on either of those two buttons, it'll show the books in the main memory or SD card of the ereader. Have a look if you like, there might be stuff in there already or there might not, and then go back to Library. |
| Now that we're done with the initial setup, the process for getting and reading ebooks goes like this: | Now that we're done with the initial setup, the process for getting and reading ebooks goes like this: |
| - Download a book to your computer's hard drive; | - Download a book to your computer's hard drive; |
| - Bring the book into Calibre; | - Bring the book into Calibre;e |
| - Use Calibre to decrapify the book; | - Use Calibre to decrapify the book; |
| - Send the decrapified book to your ereader and find a comfy chair. | - Send the decrapified book to your ereader and find a comfy chair. |
| |
| |
| ==== Getting books into Calibre ==== | ===== Getting books into Calibre ===== |
| |
| In Calibre, click on "Add Books," the green icon in the top left. A file chooser window appears. Navigate to wherever you downloaded your ebook files to, probably your Downloads folder. Highlight the book or books you want to add (holding down the Shift key is handy for selecting many books at once) and click Open or press Enter. | In Calibre, click on "Add Books," the green icon in the top left. A file chooser window appears. Navigate to wherever you downloaded your ebook files to, probably your Downloads folder. Highlight the book or books you want to add (holding down the Shift key is handy for selecting many books at once) and click Open or press Enter. |
| |
| |
| === If a book's metadata is messed up (and if you care enough to fix it) === | ==== If a book's metadata is messed up (and if you care enough to fix it) ==== |
| |
| Your ereader doesn't show you filenames, it only shows you title and author, so it's useful to get those right. If the book is part of a series and you fill in the Series information, then it'll be added to the Collections page of your ereader's menu, which can be nice and helpful. | Your ereader doesn't show you filenames, it only shows you title and author, so it's useful to get those right. If the book is part of a series and you fill in the Series information, then it'll be added to the Collections page of your ereader's menu, which can be nice and helpful. |
| To change any of this data, right-click on the book in Calibre's list, choose "Edit Metadata," and then "Edit metadata individually," and there'll be options where you can fill out any information you like, or even download information and alternate covers from the internet. | To change any of this data, right-click on the book in Calibre's list, choose "Edit Metadata," and then "Edit metadata individually," and there'll be options where you can fill out any information you like, or even download information and alternate covers from the internet. |
| |
| == Quick fixes to the most common metadata screwups == | === Quick fixes to the most common metadata screwups === |
| |
| * **Author and Title reversed** - Sometimes you'll download a book called something like "J.R.R. Tolkien" by The Lord of The Rings. The title and author sometimes get swapped round by mistake. This happens often enough that there's a dedicated "Swap Author and Title" button which can be found to the left of the Title field, it looks like a blue box and a green box doing a dance. | * **Author and Title reversed** - Sometimes you'll download a book called something like "J.R.R. Tolkien" by The Lord of The Rings. The title and author sometimes get swapped round by mistake. This happens often enough that there's a dedicated "Swap Author and Title" button which can be found to the left of the Title field, it looks like a blue box and a green box doing a dance. |
| * **I don't have a cover (or my cover sucks and I want one from a different edition)** - Just to the right of the cover image is a submenu headed "Change cover" with some buttons beneath. You can try the "Download cover" option which will automatically search Google Images and Amazon and some other places, and then choose the cover you want. This depends on your Title and Author being set correctly. If you don't see any you like, you can search in your web browser for a new cover image, save it to your hard drive, and then select it using the "Browse" button. | * **I don't have a cover (or my cover sucks and I want one from a different edition)** - Just to the right of the cover image is a submenu headed "Change cover" with some buttons beneath. You can try the "Download cover" option which will automatically search Google Images and Amazon and some other places, and then choose the cover you want. This depends on your Title and Author being set correctly. If you don't see any you like, you can search in your web browser for a new cover image, save it to your hard drive, and then select it using the "Browse" button. |
| |
| == Title and Author are really all you need == | === Title and Author are really all you need === |
| |
| The only metadata that really needs to be correct is the book's title and the author. If you want to be meticulous about things like setting the publisher and the edition number and the date of publication then that's fine, a lot of people take delight in curating these things and making their library neat and tidy. Your ereader will still read the book either way, so fill out as much information as brings you joy. If you're the meticulous sort and you already have a folder full of a thousand epubs, avoid the temptation to import them all at once; bringing them into Calibre one or two authors at a time can help keep this fun rather than overwhelming. | The only metadata that really needs to be correct is the book's title and the author. If you want to be meticulous about things like setting the publisher and the edition number and the date of publication then that's fine, a lot of people take delight in curating these things and making their library neat and tidy. Your ereader will still read the book either way, so fill out as much information as brings you joy. If you're the meticulous sort and you already have a folder full of a thousand epubs, avoid the temptation to import them all at once; bringing them into Calibre one or two authors at a time can help keep this fun rather than overwhelming. |
| |
| == Series information is not super-necessary but nice to have == | === Series information is not super-necessary but nice to have === |
| |
| The PRS-505 has a "Collections" menu that can show books by series, and this can be handy. For an example from my library, I've got a twelve-volume collection of the complete short stories of Theodore Sturgeon; if I were to leave volume 2 named "The complete stories of Theodore Sturgeon volume II: Microcosmic God" then when I browsed my ebooks by title, the ereader would truncate these massive titles to something like "The complete stories of Theodor..." repeated twelve times, and that's not ideal because I can't tell which one's which. | The PRS-505 has a "Collections" menu that can show books by series, and this can be handy. For an example from my library, I've got a twelve-volume collection of the complete short stories of Theodore Sturgeon; if I were to leave volume 2 named "The complete stories of Theodore Sturgeon volume II: Microcosmic God" then when I browsed my ebooks by title, the ereader would truncate these massive titles to something like "The complete stories of Theodor..." repeated twelve times, and that's not ideal because I can't tell which one's which. |
| |
| |
| ==== Decrapifying ePubs ==== | ===== Decrapifying ePubs ===== |
| |
| From here on out, we assume that your ebooks are in .epub format (unless you wanna read comics/manga, see further down for a special section on that). | From here on out, we assume that your ebooks are in .epub format (unless you wanna read comics/manga, see later on for a special section on that). |
| |
| Old ereaders choke on new ebook files because new ebook files are full of unnecessarily massive images and font files that old ereaders don't have enough memory to figure out. Calibre can fix all those problems and give you an .epub that will work reliably and load fast. Loading fast is important because the CPU takes so much more power than the screen - this can make the difference between going days between charges, or weeks! | Old ereaders choke on new ebook files because new ebook files are full of unnecessarily massive images and font files that old ereaders don't have enough memory to figure out. Calibre can fix all those problems and give you an .epub that will work reliably and load fast. Loading fast is important because the CPU takes so much more power than the screen - this can make the difference between going days between charges, or weeks! |
| |
| |
| === Step 1: Resize the cover === | ==== Step 1: Resize the cover ==== |
| |
| Highlight the book in Calibre that you want to convert. The cover image will appear in the right-hand panel, and sometimes you'll also see some tags and possibly the blurb from the back cover. In the bottom-right corner of the cover image will be a bit of text showing the dimensions of the cover - probably a couple thousand pixels tall and over a thousand wide. | Highlight the book in Calibre that you want to convert. The cover image will appear in the right-hand panel, and sometimes you'll also see some tags and possibly the blurb from the back cover. In the bottom-right corner of the cover image will be a bit of text showing the dimensions of the cover - probably a couple thousand pixels tall and over a thousand wide. |
| Go up to the menu bar at the top of the screen, with all the massive buttons. Click on "Resize Cover." Notice that the text on the bottom right of the cover image is now 768 pixels tall (or less), and 600 pixels wide (or less). That's it, we're done. | Go up to the menu bar at the top of the screen, with all the massive buttons. Click on "Resize Cover." Notice that the text on the bottom right of the cover image is now 768 pixels tall (or less), and 600 pixels wide (or less). That's it, we're done. |
| |
| == Step 1 in bulk == | === Step 1 in bulk === |
| If you want to resize the cover on a whole bunch of books at once, you can totally do that - hold down Shift and click to select many books, then hit Resize Cover just like before, and it'll do them all at once. | If you want to resize the cover on a whole bunch of books at once, you can totally do that - hold down Shift and click to select many books, then hit Resize Cover just like before, and it'll do them all at once. |
| |
| |
| |
| === Step 2: shrink massive goofy font files and compress images === | ==== Step 2: shrink massive goofy font files and compress images ==== |
| |
| Polishing a book file will fix those goofy massive font files and compress the images inside the book. Note that compressing is, confusingly, not the same as resizing, that'll happen in the next step. Compressing means reducing the image quality, but not in a way that you'll actually be able to tell on an e-ink screen. | Polishing a book file will fix those goofy massive font files and compress the images inside the book. Note that compressing is, confusingly, not the same as resizing, that'll happen in the next step. Compressing means reducing the image quality, but not in a way that you'll actually be able to tell on an e-ink screen. |
| Once it's finished, you might well be done, let's check. Right-click on the book in Calibre, select Open book folder -> Open book folder (or press O on your keyboard) and this'll open Calibre's folder for this ebook in your file manager. You'll see two ebook files; one will have an .epub extension, and the other will have an .original_epub extension. If the .epub one is like a couple of megs or less, then it's almost certainly gonna load on just about anything - if not, then proceed to the next step. | Once it's finished, you might well be done, let's check. Right-click on the book in Calibre, select Open book folder -> Open book folder (or press O on your keyboard) and this'll open Calibre's folder for this ebook in your file manager. You'll see two ebook files; one will have an .epub extension, and the other will have an .original_epub extension. If the .epub one is like a couple of megs or less, then it's almost certainly gonna load on just about anything - if not, then proceed to the next step. |
| |
| == Step 2 in bulk == | === Step 2 in bulk === |
| Got a load of epubs that all need this treatment? Hold down the Shift key to select a whole bunch at once and then proceed as above, then click on Jobs to see how long it's gonna take and whether you've got time to make a cup of tea. | Got a load of epubs that all need this treatment? Hold down the Shift key to select a whole bunch at once and then proceed as above, then click on Jobs to see how long it's gonna take and whether you've got time to make a cup of tea. |
| |
| |
| |
| === Step 3: resize the images inside the book (if any) === | ==== Step 3: resize the images inside the book (if any) ==== |
| |
| I was putting some Goosebumps books on my kid's ereader, and noticed that they were like twelve megabytes each and it took ages to turn pages. An .epub that's just text should be like 100 or 200kb for a kid-length story, and these books aren't illustrated, so what's the deal? | I was putting some Goosebumps books on my kid's ereader, and noticed that they were like twelve megabytes each and it took ages to turn pages. An .epub that's just text should be like 100 or 200kb for a kid-length story, and these books aren't illustrated, so what the hell? |
| |
| Turns out, it was because each chapter heading was an image containing the chapter number in a spooky font, and each one of those images was so enormous that you could print it out as a poster and hang it on your wall. If we make those images the same width as the ereader's screen, then not only do we save a massive amount of space on the memory card, we save the ereader the work of loading a too-big image and then resizing it to fit the screen as we read the book, so that saves a lot of battery life too. | Turns out, it was because each chapter heading was an image containing the chapter number in a spooky font, and each one of those images was so enormous that you could print it out as a poster and hang it on your wall. If we make those images the same width as the ereader's screen, then not only do we save a massive amount of space on the memory card, we save the ereader the work of loading a too-big image and then resizing it to fit the screen as we read the book, so that saves a lot of battery life too. |
| You can now close the ebook editor window, and we're done with this step. | You can now close the ebook editor window, and we're done with this step. |
| |
| == Step 3 in bulk == | === Step 3 in bulk === |
| Step 3 can't be done in bulk at time of writing (spring 2025), you'll have to do each file one at a time, sorry. One of these days I'll write a Calibre plugin that'll do all three steps in one click and can be run in bulk, but that day is not today. | Step 3 can't be done in bulk at time of writing (spring 2025), you'll have to do each file one at a time, sorry. One of these days I'll write a Calibre plugin that'll do all three steps in one click and can be run in bulk, but that day is not today. |
| |
| |
| ===== Notes on SD card corruption ===== | ===== Notes on SD card corruption ===== |
| I've had a couple of SD cards get wonky and quit accepting or reading new books. I'm still narrowing down what, exactly, causes this to happen, but I can quite reliably make it happen by copying either **too much data** or **too many books** to the ereader in a single operation. | I've had a couple of SD cards get wonky and quit accepting or reading new books. I'm still narrowing down what, exactly, causes this to happen, but I can quite reliably make it happen by copying either **too much data** or **too many books** to the ereader in a single operation. **Update 2025-05-15:** <del>New information has been brought to light</del> old information has been dug up from a swamp of fifteen-year-old forum posts and also I've spent //far// too much time experimenting and being frustrated, see "SD card too big" below. |
| |
| === How do you know when it's happened? === | === How do you know when it's happened? === |
| SD card/Sony Reader/database/cache.xml | SD card/Sony Reader/database/cache.xml |
| SD card/Sony Reader/database/cacheExt.xml | SD card/Sony Reader/database/cacheExt.xml |
| If this process goes wrong while the ereader is writing to the cache.xml or cacheExt.xml files (or maybe if the process goes wrong while those files are open for writing) then the ereader will never send the "Okay I'm done writing here" bits to the filesystem, and one or both of these files will end up with a chunk missing off the end, which makes the whole file no good and can often cause further problems elsewhere on the SD card. Tragically, our reading positions are also stored in these files, so the ereader will forget which page we were on! We can reliably cause this SD card corruption by sending too many books in one go, or sending a few enormous books in one go. | If this process goes wrong while the ereader is writing to the cache.xml or cacheExt.xml files (or maybe if the process goes wrong while those files are open for writing) then the ereader will never send the "Okay I'm done writing here" bits to the filesystem, and one or both of these files will end up with a chunk missing off the end, which makes the whole file no good and can often cause further problems elsewhere on the SD card. Tragically, our reading positions are also stored in these files, so the ereader will forget which page we were on! We can reliably cause this SD card corruption by sending too many books in one go, sending a few enormous books in one go, or (added 2025-05-15) trying to use an SD card bigger than 4 gigabytes, sigh. |
| |
| == Too many books in one go == | == Too many books in one go == |
| == Smaller number of books, but each book is too big == | == Smaller number of books, but each book is too big == |
| I've also corrupted an SD card by combining twelve volumes of manga into one whopping half-a-gigabyte file, because I thought I was being clever. Oops. The ereader has to open each file you send it, in order to figure out from its metadata what sort of file it is, and who's the author and what's the title; it probably ran out of memory and crashed while it had its cache.xml file open. The poor thing's only got 64mb of RAM after all. | I've also corrupted an SD card by combining twelve volumes of manga into one whopping half-a-gigabyte file, because I thought I was being clever. Oops. The ereader has to open each file you send it, in order to figure out from its metadata what sort of file it is, and who's the author and what's the title; it probably ran out of memory and crashed while it had its cache.xml file open. The poor thing's only got 64mb of RAM after all. |
| | |
| | == SD card too big == |
| | Although the PRS-505 can "see" SD cards of up to 16 gigabytes, in practice once you go over 4gb of Stuff on the card, things get really screwy. This is because the reader doesn't mount SD cards the way you'd expect any standard Linux system to mount them (aye these are linux-powered!), but uses some weird thing that Sony came up with that takes all kinda shortcuts to save battery life. Since I figured this out (by repeated frustrated experimentation plus trawling long-dead forum threads) I've repartitioned my memory card to 3.9 gigs (leaving half of its space unpartitioned) to make sure that there's no way to go over 4 gigs. |
| | |
| | (how would you even go over 4 gigs of books, Dan? Well, you can fit like a thousand books in one volume of manga and Attack on Titan is really good sooo...) |
| |
| === How to avoid it === | === How to avoid it === |
| If you're sending dozens and dozens of books, or a few enormous books (comics and manga are especially bad for this), break up the Big Send into several Little Sends. Send a few books, then eject, unplug, allow the ereader to spin its circle and do its sniffing etc, then once it's done, plug back in and send more. How many? I've found thirty or so regular book-sized-books (under a meg each) or five manga volumes (like 20 or 30ish megs) in one go doesn't cause any problems. I've sent ten mangas at a time and had it be fine but, eeeehhhhhh, I don't like to. Makes me nervous. | If you're sending dozens and dozens of books, or a few enormous books (comics and manga are especially bad for this), break up the Big Send into several Little Sends. Send a few books, then eject, unplug, allow the ereader to spin its circle and do its sniffing etc, then once it's done, plug back in and send more. How many? I've found thirty or so regular book-sized-books (under a meg each) or four-five manga volumes (like 20 or 30ish megs) in one go doesn't cause any problems. I've sent ten mangas at a time and had it be fine but, eeeehhhhhh, I don't like to. Makes me nervous. |
| |
| === How to fix it if it does happen === | === How to fix it if it does happen === |
| |
| === How to recover more gracefully than that === | === How to recover more gracefully than that === |
| Take a backup of your whole entire SD card every now and then, including the Sony Reader folder with its cache files! If your SD card gets corrupted, format it and then restore the backup. Then you'll only have to start from Some Time Ago, rather than All Over Again. | Take a backup of your whole entire SD card every now and then, including the Sony Reader folder with its cache files! If your SD card gets corrupted, format it and then restore the backup. Then you'll only have to start from Some Time Ago, rather than All Over Again - and it's WAY faster to restore a SD card image than it is to send all those books again through that retro USB wire. |
| | |
| | === Update 2025-05-15: it's the SD card === |
| | Quick update on this issue: since repartitioning my 16 gig SD card down to 3.9 gigs, I've been able to send a thousand books or a hundred volumes of manga at once with no problems at all. Granted, if I do that then the ereader will spin its circle for like an hour lol. |
| | |
| | Interestingly, I was able to send the same 3.8ish gigs of books that previously caused problems - it seems that on big SD cards, issues start to crop up as you //approach// four gigs, whereas an SD card with a single 3.9 gig partition can fairly reliably get filled until Calibre tells you there's no more room (leaving maybe half a meg for the cache file to expand). |
| | |
| | The most books I've had on at once was around 1,100. [[https://www.mobileread.com/forums/showthread.php?t=30849|Here's a guy]] who put over 4,000 on, followed by two forum pages of useful technical information (after six forum pages of people bitching at him about it). |
| | |
| | ====== What else? ====== |
| | Part 4 of this guide is [[oldtech:calibre:manga|a special note on comics and manga]], and part 5 is a [[oldtech:calibre:bonus|bonus bit on how to get stories from web pages and turn them into .epub files]]. |
