Breaking the world, one application at a time
It's taken over a year but the medal for 2022 contest is finally in my hands. 🙂
This time organizers had to deal with some manufacturing issues (which took them a full year to resolve), and I had to deal with overly zealous customs and taxes office. They billed me around 20 eur in taxes and surcharges for a simple trinket. Un-fucking-believable!
I'm extremely lucky. In my 15+ years of messing with computers, I've never lost data due to HDD developing bad blocks and dying. Never! 🙂
Other people are not that fortunate. So, last weekend I was asked to look at an Acer laptop that just won't start. Windows startup screen shows up, stays for 5-10 minutes and computer reboots. Safe mode doesn't start, Alt-F10 Acer Recovery Console won't show up, nothing. At least I got Windows Memory Diagnostics to show up - and it didn't find anything wrong with RAM.
After I disabled Automatic Restart on System Failure (and waited 10+ minutes for Windows to crash), I got this nice error UNMOUNTABLE_BOOT_VOLUME (STOP: 0x000000ED):
Considering how much time it takes to get to the error, it's probably a bad hard disk.
I removed 2 screws to get access to HDD. First thing I saw was this huge scratch all over HDD bracket and cover plastic.
Apparently Mr.Awesome Neighborhood PC Repair Dude has tried to remove HDD with a screwdriver and failed. He had also broken few plastic clips on HDD cover - but who cares about those, right? At least, he did no visible damage to the electronic parts of HDD. 🙂
Let's try to attach disk to another PC and see if it's really bad.
Let me tell you, attaching it to my Windows computer was a bad idea. When disk was plugged in, Windows took 5 minutes to start. Any program took 1-2 minutes to start. To be honest, I have no idea why Windows were acting so weirdly, but hey, kids, don't try this at home! 🙂
At least I got an output from Crystal Disk Info which confirmed my suspicions - bad HDD:
On the side note, Internet is full of really stupid advices. If you suspect that your disk might be physically damaged and dying, never ever use "chkdsk" or similar tools on it! They will likely fail and/or corrupt your data even more. Make a full disk copy and try to fix data there.
Lesson learned - don't use Windows if your HDD is dying. Linux is much safer and data-recovery friendly!
After some Googling, I found Clonezilla. It's a free Linux-based software that helps with disk imaging/cloning. Reviews were nice, so I made a bootable USB with Clonezilla and tried it out.
It failed.
After enabling "Expert options" and enabling ––rescue flag, it started to do something. However, estimated completion time of 40+ hours wasn't exactly exciting. Apparently, Clonezilla/partclone is slow! I'd love to have a solution that actually works, preferrably today.
Few more Google searches later I learned about ddrescue. It's yet-another-Linux-software that can do almost anything - iff you can master its arcane command-line arguments. As their "manual" tells it succinctly:
This tutorial is for those already able to use the dd command. If you don't know what dd is, better search the net for some introductory material about dd and GNU ddrescue first.
Dude, I AM reading the ddrescue manual. What other introductory material about ddrescue should I search for? 🙁
Since ddrescue is included in clonezilla USB image, I launched bash and tried the simplest possible version:
user@debian:$ ddrescue /dev/sdc3 /home/partimag/hdimage.img /home/partimag/rescue.log
It failed with error "Can't open input file: Permission denied". Apparently, you need to use sudo. My next attempt was actually successful!
user@debian:$ sudo ddrescue /dev/sdc3 /home/partimag/hdimage.img /home/partimag/rescue.log
So, here we are, after 5 hours of running.. Estimated remaining run time is 25 minutes and it has recovered everything but 100MB of data from the HDD... Fingers crossed!
18 hours later my fingers were still crossed.. WTF?
Well... Hidden in the ddrescue manual is this great note:
The 'remaining time' is calculated using the average rate of the last 30 seconds and does not take into account ... Therefore it may be very imprecise, may vary widely during the rescue, and may show a non-zero value at the end of the rescue. In particular it may go down to a few seconds at the end of the first pass, just to grow to hours or days in the following passes.
Holy fuck, why on earth would you show "remaining time" if you very well know that it's "very imprecise"? Does it make your program go any faster? No. Does it help your user in any way? No. It just pisses everyone off.
All in all, ddrescue ran for around 48 hours - recovering 99.98% of data. There were still 45MB of non-scraped data left but I decided that it's not worth to wait 40-50 more hours to rescue mere 20-30 megabytes.
Lesson learned - reading data from unreadable sectors is really slow. Prepare to wait for days!
Recovering data is great. But what to do with the 0.02% of data that were unreadable? ddrescue log can tell you that sector 0x12345000 was unreadable - but you will have no idea which file occupied that sector. Since I'm a Windows guy, I decided to modify ddrescue's suggested approach a bit and used Windows tools when possible.
First, run ddrescue with ––fill-mode argument:
printf "BABEC0DE" > tmpfile ddrescue --fill-mode=l- tmpfile /home/partimag/hdimage.img /home/partimag/rescue.log rm tmpfile
It will take the image file and mark all unreadable sectors with "BABEC0DE" and relevant sector/position information based on the log file. The affected part of file will look like this:
You can pick whatever text you want - I didn't want to use suggested "DEADBEEF" constant, as it is much more commonly used and might actually appear in some valid files.
Second, reboot into Windows and use OSFMount to mount the created hdimage.img:
Finally you can see files and folders from the damaged disk. Now use whichever Windows tool you like to search for "BABEC0DE". In my case, there were 16 files affected - 12 videos and 4 log files. So, nothing of value was lost! 🙂
If you have Acronis or other Windows cloning software, you could use that to write HDD image to new disk. Since I didn't have any, I use Clonezilla's bootable USB and Linux standard dd command:
sudo dd if=/home/partimag/hdimage.img of=/dev/sdb3 bs=1M status=progress
After an hour and a half all the data were transferred to the new disk. Now I just needed to put HDD back into the laptop, boot up the system and run chkdsk to make sure that everything is fixed.
After 3 evenings and plenty of swear words, it's a great success! 🙂
There are two kinds of people, those who back up their stuff and those who have never lost all their data. Be smart and make sure you have proper backups! Otherwise, be prepared to spend few evenings learning Linux disk management tools and cursing their command-lines.
Till next time!
Last week MaxXor suggested that I should add HTTPS support to my blog. My existing free host (bplaced) doesn't offer HTTPS, so I decided to finally switch to paid hosting. After thinking a bit, reading customer references, I chose Active24 - and so far the experience has been overwhelmingly positive.
For now HTTPS is optional (try https://lifeinhex.com/), I'll start enforcing HTTPS in a few days after fixing all the mixed-content warnings. 🙂
Have fun and stay safe (and let me know if you notice any issues)!
I just noticed that I haven't published a full-length post for almost 2 months already. 🙁 Let me assure you, this blog is not dead, I just had a peaceful Christmas vacation. Here's what else is new..
I'm still having fun writing this blog. So, I've renewed the domain for one more year. Since I'm using a free shared hosting, HTTPS is not an option. But if you would really like to see that (or any other improvements to the site), please let me know in the comments.
As you probably have noticed, this blog is running on WordPress. And since my webhost has really tough restrictions on what can be done with shared hosting accounts, automatic update of WordPress is not possible. So, I updated the engine manually to the latest version 4.7.1.
Last night there was some short outage but it was more likely caused by maintenance on the webhost. But please do let me know if you notice any issues with the site!
One of the most requested features for Enigma Virtual Box unpacker was support for large files. I had to rewrite quite a lot of stuff to make it happen but now it works fine for files up to 4GB (and possibly more). 😉 It's finished but I need to test the unpacker properly before making it public.
Again, one of the most requested features is to support huge files (500MB+). Since the original unpacker was written in 2009, compiled only with Delphi 4 and was intended to unpack regular EXE files, I had to rewrite pretty much everything from scratch - so I'm happy to say that it's 90% done. I should be able to wrap it up in a few weeks and then make it public.
Every once in a while someone in Tuts4You asks about video-to-exe or pdf-to-exe "protections". Every single one I've seen so far has been just a snake oil. In the article I'll document the methods used by these "protections" to encrypt video/PDF and prevent user from extracting original file out of EXE. You won't believe how much time and effort goes into preparing a single technical blog post - so don't expect it to be finished any time soon. 😉
Have fun and talk to you all later!
if you come to me with a question, i won’t give you an answer.
if you ask for my advice, i won’t give you a recommendation.
at best i’ll give you a pointer, a suggestion. perhaps a direction, a map. you’ll then start walking, creating your own route, getting lost and finding your way again. you’ll see other people’s footsteps often, some old and some fresh, and sometimes you’ll walk open territories no one has visited before. you’ll meet Eureka and dance together and laugh in joy, you’ll cry with Despair, which is not a bad thing.
if you want a lesson, i won’t give you instruction.
if you want guidance, but i won’t show you.
i want you to have fun too. i won’t spoil your journey.
or maybe i just have no idea and this is my fancy excuse to slip away from the situation?
Thank you, el trastero, couldn't have said it any better.
When I started this blog, I wanted to try and see what will happen. I thought that I have so many things to say and to write about. I still do. 🙂
But as I found out soon enough, making a decent-quality blog post takes hours. And I don't have that much free time. So, I had to choose between making half-assed posts with pretty kittens and lists of "Top-X things you don't really care about", or taking my time to write a proper post about something (technical) that I learned recently - at the expense of less frequent updates.
I chose to write less often. Currently I'm managing to make one proper post per week - and I really hope to keep it up that way. Of course, the more feedback I get, the more motivated I'd be to write. So, it's all in your hands.. 😉
On a related subject, I added a simple captcha to the comment form. Looks like it's working really well to keep most of the spambots away. But if you encounter any problems with it, please let me know.