Thursday, October 29, 2009
Data Recovery Tips and Tricks
Here's some tips on using ddrescue:
1. Always use a log file
2. Start ddrescue like this the first time:
ddrescue /dev/sda1 /dev/sdb1 logfile.log (sda1 will be the C: drive, sdb1 will be
the replacement drive.) Do not get them backwards in order.
3. Once you notice a ddrescue struggling to read through bad sectors, kill the job.
In Linux, you can try pkill ddrescue, or ps -ef | grep ddrescue . Find the PID or
process id number. Then type kill -9 pid, i.e. kill -9 1234.
4. Rerun ddrescue using the -i and -s switches, i.e. ddrescue -i75G -s125G logfile.log
(start copying 125GB after the first 75GB). The -s switch is size or amount to copy.
The -i switch is the initial position to copy from. You want to copy as much of the good
parts of the drive as quickly as possible, so skip the bad parts if you can.
5. Once you have most of the drive copied, then kill the job and type the following:
ddrescue -n -d /dev/sda1 /dev/sdb1 logfile.log . Ddrescue will read the log file and
only copy what it hasn't copied. With the above command, it will try to trim the bad
sectors and recover as much data as it can without recovering the bad sectors.
6. You can then try to recover the bad sectors, but this will be VERY time consuming and
problematic. Rerun ddrescue like so: ddrescue -r3 -d /dev/sda1 /dev/sdb1 logfile.log .
7. You can then rerun ddrescue using the -n -d switches to get anything you missed, but
by this stage it is likely you've recovered all the data that you are going to get.
Google "man ddrescue" and "info ddrescue" for more information about the program.
If you need to recover an audio CD, you can use cdrdao. Ddrescue is only good for recovering digital media. Audio CDs use a different format and can't be read by ddrescue. If the audio CD has scratches, you will get skips using a normal CD or DVD burning program. I have built a GUI front end for ddrescue that works on Windows. If any one wishes to use it, you can email me at jbmoore61 [AT] gmail[DOT]com. I hope to have it on sourceforge.net or some other convenient web site shortly.
Using an Ubuntu liveCD is easiest. Just boot your system with the liveCD. Under accessories, choose terminal. Then type sudo su. The prompt will change to # which means you are essentially root. Then type dhclient to grab an IP address, or you can click on the adapter icon in the upper righthand corner of the screen. Then type apt-get update ; apt-get install ddrescue. Choose y if prompted and ddrescue will be installed into the ramdisk. You can then use ddrescue to recover and copy your Windows drive to the new hard drive. Try to get a replacement drive that is the same size as your old drive. If you get a larger drive, you may have to run qpartd or gpartd to make the new copy use all of the new drive's disk space.
Labels: ddrescue cdrdao
Why Windows Users Suffer
I realized that a lot of this pain could be avoided if Windows ran chkdsk more often. Windows only checks the hard drive's integrity when it is setup the first time. After that, it is up to the user to check the hard drive. If the hard drive starts becoming corrupted, Windows will do the best it can until a critical file is damaged at which point the operating system will likely fail entirely. I've seen a system lock up and freeze with no errors logged. When a chkdsk c: /f/r was run, bad sectors were found and marked and the computer acted normally again. But Windows will never complain about a dying hard drive, it just tries to function until it dies. I haven't had a major data loss in several years because I run Linux and the designers decided several years ago to include an automatic file system test after 25 reboots of the system. I have had a hard drive start to fail, but the damage was minimized by the file system check routine. If Windows had this feature, a lot of people wouldn't lose their important data and have such heartache. You can google "schedule chkdsk" to find ways to configure a Windows system to check itself, but when I tried to set something up, Task Scheduler balked because my normal user account didn't have enough privilege to run it, which means only an administrator account can schedule a chkdsk to run. The easiest way to run chkdsk is manually. Just go to your menu, Start -> Run -> enter the word cmd in the box and click OK. You'll get a command prompt. At the prompt, type chkdsk c: /f. You'll be asked to enter Y or N, enter y (yes). The next time you start up, a disk check will be performed. Get a cup of coffee while your computer is doing the disk check. If you get errors and it notices bad sectors, rerun the chkdsk command like this chkdsk c: /f/r . Then go to the store or doing something else for a half hour or an hour depending upon your hard drive size.
Labels: Windows disk Integrity
Sunday, October 25, 2009
I Am Sick and Tired of People Complaining About Our Educational System
The pay for postdocs has gotten better in recent years, but the problem of career advancement remains. There are too many people competing for too few jobs because the universities need the cheap graduate student labor to keep research and development costs down. That is why the universities howled when the student visa program was changed after 9/11. Businesses are no better. I was mandated to take 40 hours training per year at my last job. My performance review depended upon it as did a professional certification. Yet, my managers seldom ever let us use the new skills we learned to make our jobs easier. They certainly didn't give us raises for getting more and better certifications in IT Security. In fact, the smarter young people generally got taken advantage of by management asking them to take on more tasks for the same amount of pay and sometimes less pay. Any suggestions we made to improve operations were either ignored or languished because they fell on the deaf ears of management. Now companies import H1-B visa workers because they are cheaper than American graduates to employ, or they outsource jobs to India or China because it's even cheaper. What do economic decisions such as insourcing or outsourcing do to an educational system's graduates and the educational system itself? When a job leaves the country, the tax income from that job is lost forever. Depressed wages lead to less tax income. It's a negative feedback loop. So, economic decisions made by businesses have ramifications beyond their business spreadsheets. If our so called political leaders don't have the backbones to say no to our so called business leaders when the latter try to extort concessions and subsidies from our governments, then the rest of us who are good taxpaying citizens suffer the consequences. If we haven't learned that lesson from the latest financial crisis, then we deserve the fate awaiting us because of our misguided economic beliefs, ignorance, and shortsightedness.
Labels: hypocrites and morons in control
Wednesday, October 21, 2009
Symptoms of a Much Bigger Problem
Kajakai Dam supplies power to southern Afghanistan, but unfortunately, only one turbine is currently working. A major joint British and American operation delivered a new turbine to the dam last year, but according to Michael Yon, all that money and effort may be wasted. Without the dam's powerplant coming fully online before 2014, aid money that was contingent on the dam providing needed power for Helmand Province will not materialize.
Then, there is the US Intel Community using open (public) sources for information. They've always used such sources, but now they are shooting themselves in the foot because of analysts giving themselves away on the Internet.
There are perils to the process. One source here said that analysts who engage in searches without masking their origin can lead to foreign governments or companies cutting off access to web sites or to people involved. The problem? Some analysts at NSA, CIA and other alphabet soup agencies forget to mask their IP addresses and the times at which they are searching. Chinese, Russian and other savvy operators can check time stamps, for example. If a search occurs during American working hours, it’s a pretty good bet that it’s an American source looking for the information.
Haven't these people heard of open proxies? If Russian cybercriminals can buy and use a colocation facility in the U.S., how difficult would it be for our Intel Community to buy or create open proxies in neutral or foreign countries for their analysts to use, and then train them to use them. There may be technical reasons why this is difficult, such as the use of a mandatory proxy server at work, but that's why the Three Initial Agencies have big black budgets. There are workarounds for these sorts of issues. But, that doesn't let them off the hook for letting their analysts get sloppy doing their research and compromising their intelligence gathering.
This last blog post by Tom Ricks is from a Canadian Military Intelligence officer who recently came back from Afghanistan. His report is illuminating and would be hilarious, verging on hysterically funny, if it wasn't symptomatic of the underlying problems NATO faces training the indigenous forces in Afghanistan.
Afghan National Army military intelligence officers brought an interesting perspective to signals interception: "rather than passively listening [to enemy radio traffic], the ANA had a tendency to get into arguments with insurgents."
Yet, when someone believed in us and did their job, we abandoned them to the Taliban.
In one remote village, strong Afghan commanders worked hard to deny the area to the Taliban, and also gained a remarkable amount of intelligence. But then the outpost "was closed just after the end of our tour due to its sustainment difficulties, in all likelihood dooming many of the locals who had collaborated with us there." This is the opposite of protecting the population -- it is endangering them.
These issues shouldn't have happened. They are all easily preventable. That they have been allowed to happen and continue unchecked shows a lack of attention to details that matter. All of these issues are symptoms of a much deeper and worrisome problem. With Afghanistan, they indicate that we will fail because we are not carrying through on our strategies and objectives. If we don't care about the Afghans and their country, then what are we doing there wasting Afghan, American, and others' lives, money, and resources? In the case of intelligence analysts being allowed to compromise their own online research, it implies that the American Intelligence Community isn't nearly as smart as it thinks it is. I hope the Chinese and Russians aren't laughing too hard while they collect reams of intelligence (and money) from our own military and civilian networks while denying us the same.
Labels: we are in trouble
Monday, October 19, 2009
Confusing the Nmap Scanner
1. Create a firewall rule that drops all packets recognized as coming from a port scanner.
2. Change the strings in the source code and recompile the application.
3. Change the strings in the binary file itself.
In my case, option three was the easiest option because the system I was interested in protecting is a honeypot in a DMZ. Option 2 may or may not work depending upon how old the source code is and what compiler and other build software it requires. Anyway, I fired up khexedit and replaced the version number and OS version characters within the sshd binary. You'll want to try to avoid adding or subtracting characters and just replace them with different characters. After testing, OpenSSH appears to be working fine. I also did the same thing for the vulnftpd.so module for nepenthes. The first time I tried to change the string in the module it broke nepenthes, so you might want to back up the file you are working on before you try this. When I ran a nmap scan on the honeypot, nmap failed to recognize my openSSH version or that nepenthes was running on the box. This trick won't fool a sophisticated attacker, but it might confuse a script kiddie or an automated nmap scan.
Saturday, October 10, 2009
Simple Gimp Tricks
Here's another still taken from the Flip video mp4 footage I shot:
opossum in the dark - an original snapshot
Here's the level corrected version:
That's a bit better. The corrected jpeg picture is closer to the light levels I see in the actual digital movie. Uh-oh! This posting looks fine on my old Linux workstation, but on my laptop, the upper photo is fine, but the lower photo is too light. Guess I should have checked the posting on all of my systems to see if there was any difference in light levels.
Friday, October 09, 2009
Obama Wins the 2009 Nobel Peace Prize
This blogger has a point. All the wingnuts on Fox News and elsewhere are going to explode, so there is a humorous side to this announcement. Considering the bombastic vitriol on Fox News and talk radio though, this isn't a major insight.
Labels: 2009 Nobel Peace Prize
Web Page Text Smeared When Scrolling Down
Still having problems with the X-server crashing. I had to remove the nomodeset boot parameter in menu.lst. Performing the following:
apt-get install fglrx-driver
likely resolved the issue since the fglrx-driver is for 2-D graphics for ATI cards. Nothing else seemed to work.
Labels: crappy drivers
Thursday, October 08, 2009
Online F.A.C.E. Training
Labels: FACE training
You are a snowflake. Arising from nothing, perfect in form, evaporating in the bright sunshine. An endless end since you will return as a snowflake. Your essence is Stillness. It is not born and cannot die. That is the reality. What other snowflakes do, say, believe, or imagine is largely beyond your control or grasp. Snowflakes are best born and live in silence. Their silence inspires an appreciation and wonder of beauty. Snowflakes born in the fury of the blizzard inspire fear of the cold and of the elements. Each snowflake thinking itself flawed wishes to be beautiful, despite the fact that it was born beautiful, but many mistake noise, sound, and other nonsense with beauty instead of realizing that all beauty arises from silent awareness.
Wednesday, October 07, 2009
Learning Qt4.5 - Qt4 SDK and Qtcreator 1.2.9
I've been learning the Qt4 version of C++ programming for the last few weeks. I started playing with qtdesigner last spring and then dropped it. A newer version called Qtcreator came out recently, so I picked up where I left off. The relevant links are Qt4 download page for SDK, Qtcreator, etc. and bleeding edge Qtcreator 1.2.9 snapshot. I'm using The Book of Qt4 as a reference. It's not the best manual for learning an SDK, but it's pretty much all that's available. It's rather a shame since the Qt4 toolkit is well designed and it is easy for even novices with some help to code a small decent graphical C++ program or GUI front-end for a console application. There are some decent tutorials at tuxradar.com. One of the tutorials, the media player, leaves out some key information that's in the code, and I could not get the two media based programs to function properly on my x86_64 Linux laptop due to the fact that the tutorials were created on a Mac. The ffmpeg program failed because it didn't recognize a program option which was not the fault of the Qt4 application. GTK failed on the media player which was also not the fault of the Qt4 application. Qt4 is supposed to be multiplatform, but if the underlying software fails the developer is screwed unless he can find a workaround. The tutorials are still quite useful, more so than some of the book examples which are scattered piecemeal throughout the book and therefore difficult to assimilate or understand. It's difficult to capture a visual workflow or process in a book, and the toolkit is intensively visual.
With Windows, one will need to install the SDK first, then Qtcreator. It is also best to install the SDK twice, and custom install the second copy. One will need to bring up the qtconsole on the second copy and run the following commands:
mingw32-make or mingw32-make sub-src
(Info from qtnode.net and trolltech, and from sourceforge.net.)
The first make command statically compiles all of the SDK source code and examples while the second version mingw32-make sub-src only statically compiles the SDK source code and takes significantly less time to build if one is in a hurry. However, the latter command still takes 30 or more minutes on a dual-core system. You will then need to point Qtcreator (via Tools-> Options) to the static version of the SDK, though it's pretty good at autodetecting the needed files. I still had to include the mingw32.dll with my Windows programs to get them to work, but the statically compiled versions were significantly smaller (10-fold) and more portable than the dynamically linked and compiled versions.
I primarily develop on Linux and then port the code to Windows. The problem with Windows is that the application paths and environmental variables are different which means that one has to change some code (see below) unless one is smart enough to create portable code. (Conditional if or switch statements which check to see which platform the code is being compiled on can be used.) I spent 2-3 hours last night trying different variations of code portability and failed due to variable scope limitations in the conditional statements which was a bummer indeed. I ended up putting Windows specific code in comments. I will likely have to post something on the Qt specific forums to find the answers I need.
Coding differences with Linux and Windows:
QString program = "program name";
QString program = "absolute path of program";
e.g. QString program = "C:/Tools/scalpel/scalpel.exe";
args << "-c";
args << "C:/Tools/scalpel/scalpel.conf";
So, I have to add two extra lines and modify a third line of code to get the same program to work on Windows. This is not even taking into account differences in the Windows version of the program my program is accessing.
Addendum: Found instructions for installing and running the latest version of ffmpeg and x264 on a Debian based systems. There is also this ffmpeg cheat sheet. The ffmpeg front-end program now works when built on my AMD64 laptop.
Labels: Learning C++ and Qt4 SDK
Sunday, October 04, 2009
How Many People Will Fall for this Phish Email?
I received this in my inbox tonight. I wonder how many people will fall for this ruse? If you look, it seems to come from support.com. Likely support.com's email server was used as a relay. This is support.com:
Administrative Contact, Technical Contact:
Vatturi, Sujan Sujan.Vatturi@support.com
1900 Seaport Blvd
Redwood CIty, CA 94063
Any real email would come from a Bank of America address, and they would not have you fill out a form to send them via email or post via the Worldwide Web unless you were a business account. (Yes, business to business transactions are a lot more lax. You'd think the security would be tighter, but it's actually almost nonexistent in many instances.) I generally don't comment about these things, but these scams are becoming so common and banks are losing people's money to these scams with greater frequency, yet no one knows the losses. What's worse is that your helpful Windows computer may be the link in the chain that allows some bad guy to rob your bank account some day. I don't use Windows for any banking transactions and I've had one credit card account compromised this year which means that either a merchant I've used, the bank that issued the credit card, or the company handling credit card transactions had its systems compromised. My guess is the latter since I didn't use that card for very many purchases.
But back to the scam at hand, buried in the web form they have you download is this bit:
...="Ps" action="http://220.127.116.11/services/bofa.php" method="post"
18.104.22.168 is in Lima, Peru.
IP Telefonica del Peru
Calle San Felipe 1144, 1144,
LI34 - Lima - LI
So, you are submitting your information to a server in Peru, but when you connect to it, it immediately redirects your browser to Bank of America:
wget -c http://22.214.171.124/services/bofa.php
--2009-10-04 23:57:42-- http://126.96.36.199/services/bofa.php
Connecting to 188.8.131.52:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://www.bankofamerica.com/state.cgi?section=generic&update=&cookiecheck=yes&context=&override_debug_mode=DEBUG&template=rv_loans&type=&destination=nba/vehicle_and_personal_loans/index.cfm?adlink= [following]
--2009-10-04 23:57:42-- http://www.bankofamerica.com/state.cgi?section=generic&update=&cookiecheck=yes&context=&override_debug_mode=DEBUG&template=rv_loans&type=&destination=nba/vehicle_and_personal_loans/index.cfm?adlink=
Resolving www.bankofamerica.com... 184.108.40.206
Connecting to www.bankofamerica.com|220.127.116.11|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
The 302 code is an HTML redirect. They don't want their site indexed by search engines because this is the easiest way to find them and shut them down. However, if Bank of America was clever, they would be correlating their customers' IP addresses accessing their web server with redirections from 18.104.22.168 to gain a rough idea who is being fooled. They should also put in a complaint with that Peruvian telecom to shut down that server. But all the customer sees is the following web page while the bad guys abscond with the financial information the customers have given them.
Redirection to Bank of America from Peruvian server
Pretty clever, huh? It's an easy way to make a living for someone in a poor country. And, it costs all of us money in the form of bank fees.
Labels: Crappy IT Security
So, if shared experiences are the most common events between human beings, how can a religious belief system work by emphasizing that people have personal experiences? Personal experience seems to be an oxymoron. Krishnamurti uses the ocean as an example and says that the ocean is there for any one to look at and admire, but it is not YOUR ocean. You don't own it, so it can't be a personal experience. But the same holds true for most everything else on the planet. The only people who may have had a personal experience were the command module pilots on the Apollo Moon missions who orbited around the Moon alone while their colleagues were on the lunar surface, and they weren't entirely alone except when their spacecraft went into the radio shadow of the Moon.
I believe that what Krishnamurti and other people are emphasizing is that spiritual growth is the lessening of the "me", the "me" identity of the mind. Others call it the loss of self. When I was born, there was no "me" - no conscious "me". There was a baby, but there was no "me" yet. When this body dies, there will be no "me" as well. The mind will have died with the brain. So, what is it that strives to understand whatever it is? What is it that loves, knows joy, laughs, cries, and tries to find happiness and contentment in the here and now? What is it that recognises itself in others, but is not the "me"?
Labels: shared experience