Threat Modeling with Customers as Assets

One thing I've been contemplating lately is if the security world isn't missing something when threat modeling.
Threat modeling will generally list assets and then see what kinds of threats can threaten those assets and see what the expected loss might be.  Assets are typically thought of as being physical systems like a web server or database. 

But a consideration has been percolating since I had a discussion with one very confident threat modeler working for a fortune 100 company.  I suggested that it would be possible to consider a customer, who has a company distributed token and can move their data from one service to another in seconds, could be considered an asset.  This credentialed expert was absolutely beside himself with indignation, "the business doesn't own the customer. And a customer is also not an asset because a customer can't be attacked or exploited!" 

Oh really?

Follow me down this rabbit hole and see if there isn't a case to be made for at least considering that your customers might be an asset.

An asset is defined as:
"Any item of economic value owned by an individual or corporation, especially that which could be converted to cash. Examples are cash, securities, accounts receivable, inventory, office equipment, real estate, a car, and other property. On a balance sheet, assets are equal to the sum of liabilities, common stock, preferred stock, and retained earnings. From an accounting perspective, assets are divided into the following categories: current assets (cash and other liquid items), long-term assets (real estate, plant, equipment), prepaid and deferred assets (expenditures for future costs such as insurance, rent, interest), and intangible assets (trademarks, patents, copyrights, goodwill)."

As a business do you invest money to retain customers? - maybe you don't own customers but you invest in them -
As a business do you carefully maintain and grow your customer base?
As a business do customers provide you money? (maybe not unlike accounts receivable?)
As a business, if all of your customers leave do your physical assets have any value beyond their depreciated physical value?

So I think a reasonable person would agree that customers are an investment which pays dividends over time.  Also note in the definition of asset the sub list of "intangible assets" which include... goodwill.  Your customers represent a huge pile of cash which may or may not be delivered to your revenue stream. (think Netflix when they changed their terms and service model)

So let's say for the sake of argument that customers are the embodiment of goodwill represented in terms of accounts receivable and have been invested in both in terms of advertising dollars and education as you have tried to help your customer keep themselves secure while they use your product.

Let's go further and examine this claim that a customer cannot be attacked by a hacker.  I would suggest that it's not only possible but it is part of your threat model and sphere of business.  It is your concern.  My justification is as follows:

1) customers can be socially engineered
2) customers can be attacked by spam, trojans, viruses
3) customers can be lured away if service is denied from the customer's computer
But when someone tells me that something is impossible the lateral thinker in me feels compelled to find an exception to the rule... what if the customer is attacked by DOS and thereby denied service and leaves your service because it never works?  What if a browser helper object is put on the user's browser that denies service to Amazon Web Services or Netflix or Gmail?  What if a BHO stops service to one company and facilitates it with another?  I can think of one such BHO - Cool Web Search. So such a thing has already happened.  Microsoft has already realized they they need to protect their customers via anti-spyware and anti-virus in order to keep their company alive.

So your customers can be threatened!  The customer can take their money elsewhere.  If compromised, the customer could lead to a general compromise of your entire system. The following source indicates that customer data is an asset and probably could be considered even when transient. http://pentest-standard.org/index.php/Threat_Modeling  Consider for a moment what happens when the customer has tiny bit of our data such as an access key or certificate?

Could you educate your customer about how to keep their secret key secret? 

Could you educate your customers about how to avoid being socially engineered? 

Could you educate them on how to keep their computers secure? 

Could you educate your customer on how to avoid service interruptions? 

I think the answer to all these questions is yes even if it would lead to an extra expense...  You probably already have spent money trying to educate your customer to some degree.  So maybe you can't entirely control your customer but the customer can be educated and controlled, and even somewhat protected.  * almost like an asset *  You could provide your customer with a security token for example, or a X509 certificate.  Maybe you could even put a program on their computer that assists them in securing their credentials.

Now you can say that you're not responsible for the customer... and why should you be?  Why take on an extra expense for something you can't control?  Given the threats we face you may want to consider changing your thinking.  Operating system vendors like Microsoft understand that they need to make efforts to secure their customer in order to keep them.

The following is a hypothetical which may illustrate the point-

What happens if a customer uses your company's lowest assurance method of login to your cloud hosting services... a shared key for example.  A hacker uses Google to find the customer's exposed code using a specialized search.  Imagine then, the hacker uses the customer's login and secret key to create a fully authenticated message to the customer's account gaining access to a VM that can't be traced back to them.  Then the hacker uses the new zero-day ring 3 VM breakout to compromise the host running hundreds of VMs inside the cloud service.  From there, the hacker or nation-state can leverage sufficient computing power to crack other systems in minutes.  The hosting company now has a compromised reputation, and permanently compromised revenue stream all because they didn't consider their customer is an asset.

So should a business fold users into their threat model?  The answer I would say "depends".  Customer=Asset does not compute for every industry.  But if your customers have highly portable data that can be taken elsewhere and and your business' reputation for security is the only thing between it and disaster... including your customer as an asset in your threat models may be the only sensible approach.

DC

Extreme Browsing

TAILS

If you are testing security or you simply want to surf without tracks you might enjoy TAILS which is a bootable DVD running Debian.  Essentially this will allow you to turn most any machine into a completely safe virtual browsing envrionment.  Check out the download here: https://tails.boum.org/download/index.en.html


Surfing "Home Style" with Socks

SSH tunneling for a home-away-from-home experience.  Ever find yourself behind a corporate firewall or untrusted network and want to check your personal email or your facebook account to see what you need to bring to the party tonight?  If you just need to browse safely you can easily do this via SSH and get your encrypted session from wherever you are to home.  Your home computer will then make requests on your behalf and pass the data back to your browser in an encypted session. If you're checking email you can usually do this via the web as well even on your POP accounts as most POP providers have browser front-ends you can access if you have the URL.

How do you set up this little magic trick and surf stright though firewalls and super IDS/IPS systems?

1) Set up a linux box at home with port 22 (SSH) incoming and outgoing on your firewall.  (Make sure the account has limited privileges and a very strong password)  If you know what you're doing you may even want to set up certificates.  

2) If you're on linux you're good to just open a shell and SSH -D (see the detail below) to your home machine and skip to step 3.  If you're on Windows you'll need Cygwin or Open SSH etc.  I have a small Windows app that I wrote to accomplish this in about 800KB if you want to set up on Windows and don't care to install a bunch of stuff.  [If you comment with your email I can send you a copy]

The popular thing is to use -N -L and pick your ports but there's a far easier way for the browser using built-in Sox proxy capability. Using a Sox proxy you can shorten your SSH command and save yourself some head ache and configure your browser more easily.

Execute on your client machine:

ssh -D 9050 [username]@[remote.server.name]

(remote.server.name is the IP or address of your linux box at home which is running SSH)

Your tunnel awaits!  Your session will expire eventually but while SSH is connected you will be able to set up your browser and surf safely using the port you chose.

3) Set your browser to use a Sox proxy at 127.0.0.1 port 9050 (or whatever port you want to default to.  Type in your destination into the browser such as:  http://www.ithumbmynoseatsensiblesecuritypolicies.com and voila you're surfing around all sensible precautions and filters.

You can set a specific browser instance (say Chrome or FireFox or IE) to always connect Socks.  This way you can keep your regular browser normal and playing nicely through the firewall and then open your browse-by-home super secret browser and surf like you're on your home network.

Of course this violates all corporate policies we know and love.  It also gives you a taste of what can be done if you let your home network or corporate network get hacked.  You shouldn't use this if you intend to stay compliant with any sensible regulation or policy.  But in a pinch this will get you home-style access from behind the great firewall of China.

You are what you surf so be safe.

DC

Yahoo and Facebook team up to snarf usage data

Yahoo now requires you sign an agreement when you use their comment system to leave comments on political news.  The agreement gives them the right to use any Facebook data they deem necessary and naturally they exclude themselves from all liability in taking and using whatever data they want.  If you haven't already clicked through the agreement blindly you may want to check it out and read the agreement text.

Yahoo and Facebook use "nonce" which is a one time usage key assigned to a user for a specific session which generally changes with time.  This is to stop cookie replay or man in the middle attacks but do nothing useful against "boy in the browser" attacks or XSRF.  XSRF is particularly useful in exploiting sites like Facebook, Google, Yahoo etc. which keep you logged in 24x7x365 unless you explicitly log out.  Facebook thoughtfully keeps you logged in even when you log out because they know you really don't ever want to leave...

The downside with sites like YouTube/Gmail and Facebook is that if you poke around Anonymous posts long enough you will find that they are actively exploiting your perpetually logged in status and the nonce system does nothing to stop your browser from making the evil request on your behalf. I found a link posted by an Anon member which opened up blogger.com and activated the change-password page.  But this issue illustrates a point with the danger of the perpetual login-session which all of the big-brother style systems are implementing.  When sites keep you perpetually logged-in in order to watch your behavior and to provide "convenience" then they are keeping themselves open to the possibility of a world-wide breach which could be exploited quickly to reach 100% of systems that view the poisoned page.

At least with a boy in the browser type of attack, attackers are limited to doing what Facebook and others already do... gain personal info about you or using your account for spam.  There is a small measure that sites can provide which stops the boy-in-the-browser attack which is for the developers to keep the submit button deactivated until keyboard input is received and validated in some way.

When using these super-sites you would be well advised to log out of them before going on a surfing session.

.Net IDE Error 9009 on compiling a project with post-build commands

The compile error 9009 may also come in the flavor of error 22 or error 1 on projects with a post-build command.  Sometimes it will even have multiple errors on the same item.  What is happening can be generally found by cutting the post-build command out of the error summary at the bottom of the IDE and pasting it into notepad, removing the prefix and postfix data and then trying it in the command line dos-box.

Generally there are three main things that will mess up post-build commands...
1) bad path
2) a path with a .,:=[ ] in it
3) a path with a space in it

Because DOS 8.3 naming conventions still live under the covers of post-build command execution you can expect to get hamstrung by any of these scenarios.  If you're developing under your personal workspace in windows you likely are being affected by number 3.

There is an easy way to deal with each of the above problems.
1) check your paths by pasting them into a File Explorer window
2) replace any of those characters with an underscore _
3) put your paths (even system named path variables) in double-quotations. "$(MyPath)\bin\debug\" for example.

Installing a .Net app as a service without using installutil

Possibly the easiest way to install a .net service without needing to have installUtil.exe on the server is to open a command prompt as Administrator (right-click the command prompt menu item and pick Run As Admin)

Once your command prompt comes up you can simply enter the following command line editing in your service name and path referencing the following example.  (note the space after the = sign is critical)

sc create MyServiceName binPath= "C:\program files\My Service\MyService.exe"

Convert Int64 to Binary

I had a situation where I needed to convert an Int64 to a Sql.data.linq.binary and just could not find any valid posts on the net for how to make it happen.  Those that I did find had mistakes in their implementation and either didn't compile or were converting to a base64 which is out of this universe wrong-headed.

I solved it by combining combining two wrong answers to make a right answer.  (And they say that "two wrongs don't make a right" but I've heard that three left turns do!)  Here's the real way to do it...

System.Data.Linq.Binary myBinaryVar = null;

myBinaryVar = new System.Text.ASCIIEncoding().GetBytes(Convert.ToString(myInt64Val,2)); 

.NET 4 Assembly Security's Fresh-New DLL Hell

If you've been installing applications that use the .net 4.0 framework you probably have seen this message:
------------------
Problem signature:

Problem Event Name: CLR20r3
Problem Signature 01: cps.fuzzypreproc.exe
...
Problem Signature 04: Cps.FuzzyPreProc
...
Problem Signature 09: System.IO.FileNotFoundException
OS Version: 6.0.6002.2.2.0.272.7
----------------

If you're like me, you first checked to see if all your assemblies were copied locally and that your .exe permissions were appropriately set.  When you discovered that all these things were in order you probably started searching the net only to find millions of hits describing how this error has plagued developers in every form or .net for different reasons over the years.  Basically your Google results are a misleading spam of old problems and solutions.

If you have this problem AND are building with Visual Studio 2010 in .Net 4.0 you will experience this error for a different reason than all the other generations past that have suffered with it.  (The security requirements of .net 4.0)  .Net 2.0 used to use something called security "transparency" which allowed the .net 2.0 clr to find a matching dll and load it no matter if it was in the GAC or not.  No doubt this was done to ease the pain of .net developers competing against fast and loose java deployments which require little more than pushing a cleverly disguised .zip file to the server.  Now that security is being built into VS2010 more seriously we find that strongly named assemblies need a little more care. 

Specifically if you have a strongly named assembly and are design-time binding it. Reffing it directly in the designer you will be forced to install it into the GAC.  This means that your installer must be run locally on the server and the assemblies will be loaded into the GAC as trusted assemblies with appropriate permissions.  But your app will no longer find assemblies in it's local directory just by virtue of being there.  This new change is designed to resist hacking by keeping a malicious software developer from dropping a strongly named assembly in the local directory of an application and have it get picked up by the exe when it runs next.  Even though this would likely require a trusted access to a server (maybe even admin access) Microsoft is trying to lock down the deployment environment against tech savvy attackers.

If you have seen this bug, chances are it's been with a frequently updating tool set like a Telerik or some similar GUI tool set that tries to keep up to date with the latest compilers.  The problem does not seem to affect assemblies built with former versions of .net even if they are used under .Net 4.0. 

Solution:
So what do you do to fix this problem?  Microsoft suggests that you can use the solution in the post below, allowing for apps to use CLR 2.0 conventions if necessary, or you can run the client install of your tool set on the server (... i know... probably forcing you to fork out more dollars to the vendors and causing you more licensing problems) or you can build your own installer and digitally sign it.  A common suggestion is configuring a click-once install but there are permissions problems with it if installed remotely and you still have the problem of getting your assemblies into the GAC.  So I'd recommend the 2.0 support only if other solutions are impractical.  You may want to run as administrator and open Explorer to c:\Windows\Assembly\ and drag your files into the GAC.  But good form probably dictates having a signed install if you're going to be doing this regularly.  But once the strongly named assemblies are in the GAC this problem will magically disappear.

'Error initializing queued image'

Working with Faxman you may receive the following error: 'Error initializing queued image'.  This error occurs when faxman is trying to create an appropriate outgoing image for a fax document usually using a .FMF or a .PDF. 

If you are using a .FMF file then likely the error is an issue of access.  For example you are running the faxman service under a "Local" account and the file resides on a share. 

For most other cases you may have the incorrect file path, incorrect file name or wrong file type!  Check your file type to ensure it's coming in as a proper .FMF or proper .PDF file.  Put a debug line in your code just as you add the filename to the fax.  Copy the file path out and paste it into your file browser to ensure it's valid.  Dump it to MadEdit or another binary capable text editor to take a peek at what's inside.  Chances are if your path is correct then you will discover the file type is a standard tiff (not a FMF) or an incompatible 3rd party type.

Happy debugging,
David

A fix for using Faxman under Visual Studio 2010

If you have run into the following error as pictured below:
Mixed mode assembly is built against version 'v1.1.4.122' of the runtime ... while using the faxman component under Visual Studio 2010 and .NET 4.0 there is a work-around.

 Open your app config.  Your startup segment should look roughly like the one below.

Change the opening startup tag to contain the "useLegacyV2Runtime..." element.

Adding this line should get you building and running in no time until Faxman comes out with a newer implementation.

I hope this can get you working again.

David Cross

Software Death Cycle and the Real Cost of Your Software

Software projects continue to be developed 24x7 in every country in the world.  The software shop or internal development group always vociferously argue that their project will save money.  Millions of dollars will be typically spent.  But what is the real lifetime of that software?

The answer is 2 years.

The reason for this varies but here are the contributing factors:
1) business requirements change
2) technologies change
3) software never meets the functionality need because it's late and or not finished
4) another group claims that they can replace the software with added efficiency using "new" technology and more strict methodologies in less time and with lower cost

Business requirements change over time and this inevitability leads directly to 4 on the list.  How much money do you spend rebuilding the same software every two years?  Why does new technology always justify a complete redesign? 

Vista Drivers on Mac OSX (boot camp)

Several iterations of Mac OS have allowed dual boot capability with Windows. Essentially you can partition your hard drive using the boot camp program and allocate space to other operating systems.

Installation of Vista on using bootcamp:

Follow the prompts through creating your partitions.

You'll want to allocate at least 40+ GB of space for Vista. More space is required obviously if you want to add significant programs to it.

**To install Vista put the Vista disk inside BEFORE you click "Start Install" otherwise bootcamp will crash.**

Allow the Vista install to start.

Pick the partition you want to install Vista on.

Vista will complain about the partition not being the correct type... click on Advanced or Options on the Vista install screen and then click "Format". Vista will then do what it should have done in the beginning and prepare the partition and will install it's files.

Starting the Operating System You Want:

Restart your Mac and hold down the Option key as the screen goes dark during reboot. This will trigger the bootcamp menu and it will list the installed operating system that are available to boot to.

You can change your default by booting to your Mac side and changing the default in your system options bootcamp section.

Inevitable Driver Problems:

One of the lingering problems however has been the issue of drivers for Vista. The internet is filled with all kinds of bad advice on this topic and you are likely to get in trouble if you dig around too long. Before you waste any time on that try the easy way:

1) boot to your Vista side
2) put your Mac OSx CD, or bootcamp CD in if bootcamp comes on a separate disk (generally it will be the OS install disk itself)
3) it magically installs all the drivers you need
4) reboot

That's it!

If you follow this advice you will save yourself (as I can attest) hours of technical support and hours and hours of surfing the web and trying poorly informed advice from message boards after Apple technical support says that this is a "Microsoft problem" and you should talk to them instead.

I do know from experience that Microsoft USED to support these drivers and removed them from the Vista install (I wonder why?) So it is BOTH an Apple and Microsoft problem. Let's hope Apple tech support reads this blog and finds out how to install Vista properly using Boot Camp.

Humanizing Software

Recently we've begun working with Ultimus, a BPM system which I believe is on the verge of revolutionizing software development. Does this mean Business Process Management software is ready for prime time? Yes and No. Some issues with new versions of Windows keeps it off of the new gold standard Windows 2008 Server. Other issues with form submission that will negate form data entries unless you click in the form prior to submitting it. Speed of course is relative but when you're talking the speed of Ultimus versus phone, sneaker-net or IM you're still talking a big time savings.

Where I think BPM shines is in queueing work flow and in creating a manageable people process around data.

Instead of hard and fast rules coldly analysing and rejecting people or charging them extra money etc, why don't we involve humans in the process at critical decision points? Instead of creating a log jam effect I believe it's possible to send "threshold" cases for human review. This gives you the efficiency of blindingly fast determinations with questionable decisions being routed to a human gaining the best of both worlds. I believe that quality software in the future will ensure human interaction instead of avoiding it.

Active Directory Tools

I just found one of the sweetest freeware tools for recovering deleted objects in Active Directory. http://www.quest.com/object-restore-for-active-directory/

Quest also has other helpful freeware for Active Directory among other things.

SQL Magic Part 1 - Select Distinct Tricks

Select Distinct is possibly one of the most useful SQL tools but one of the most flawed. Select Distinct will give you only 1 column of data. Why is that? Shouldn't it be able to do a distinct selection on a target column and also return the other row's columns? One would think so but this is not the case.
If you need to grab all columns out of a table while doing a select distinct you can try something like this:
SELECT * FROM DUPES AS D3 WHERE 1= (SELECT COUNT (*) FROM DUPES AS D2 WHERE D3.ID=D2.ID AND D3.NAME <> D2.NAME)
This gives you all the records (and all columns) where there is only 1 instance of an item. You can adjust the 1 = ( Select to a 2>= to get all items that have 2 or or fewer instances of the data you are looking for.
You can do a multi-column distinct query using a sub-select for example:
SELECT DISTINCT (ID) FROM DUPES AS D3 WHERE 1= (SELECT COUNT (*) FROM DUPES AS D2 WHERE D3.ID=D2.ID AND D3.NAME = D2.NAME AND d3.MEMBER = d4.MEMBER)
Of course we could do even more fun stuff given another level of sub-select query but SQL will not go more than 1 level in sub-selects. If we could do a two level sub select we could return all columns for a given distinct query by appending a select * from dupes where 1= (select distinct.... (select count(*)...))
Granted we can write stored procedures to do some of this stuff programmatically but deficiencies in the Distinct function are significant and cost a great deal of time in work-arounds. If new SQL standards come out any time soon they should include a Distinct function that allows for a multi column distinct specification AND wildcards.... ex: SELECT *,DISTINCT (ID, MEMBER_KEY) FROM... or perhaps better... SELECT * FROM DUPES HAVING DISTINCT(ID,MEMBER_KEY)

Herding Cats

Managing developers is like herding cats. Developers are skittish, fickle and smart enough to be dangerous. Every programmer thinks that he is the next Bill Gates which is ultimately not too far out of the realm of possibility.

With this in mind how does one manage programmers through a restructuring or other major shift in development? The answer is non-obvious to those who have never done this before but surprisingly simple...
1) Always start with building trust. This means that you have to know what you're doing.  It also means that you need to invest serious time in the project.
2) Don't buzz-word drop. Everyone hears something different when people drop buzzwords and almost inevitably it will lead to preconceptions of what is being discussed. And this includes talking in broad generalities.
3) Keep discussions/meetings to the point and don't let them wander off topic.
4) Use fist-of-five decision making or use a proper Decision Analysis process with requirements and a ranking cube.
5) Be smart go-Agile. It's a foregone conclusion that old methods do not work. Agile systems represent 80% of the successful software development projects.
6) Put project management where it belongs... with a group representing each discipline within the division. Let them chose their own team to lead the project.
7) Stay out of project management... act as oversight, don't meddle
8) Drive discipline-specific pride by finding creative ways to encourage BA, QA, Dev, PM teams to take pride in what they do and build personal skill and presige.
9) Keep the team together whenever possible. Lay-offs and firings are almost always counter-productive.
10) Reward, Reward, Reward and go back to step #1.

So if you are doing a major revamp of how your company does things... give these ideas a try.

~DC

Moving MS Office to a new computer or hard drive

I recently had a hard drive drive fail. It lasted long enough however to get the License Key for MS Office and a few other handy programs out of the registry with this tool: http://www.magicaljellybean.com/beta/

It's free and is a must have if you have to move software from one computer to the other.

New Paper and Sample Code on TrustSecurity

I have posted the paper on the security vulnerabilities found in Zone Alarm, Norton Antivirus and other programs. The paper is called "In Certificates We Trust" and it has all the juicy details about how changing the system clock could cause a great many programs to stop working even critical ones like anti-virus. With the vendors notified and the products fixed the fun was over for a while but it may be time to dust off this tidbit and see what it does now. Of course Microsoft was notified and said that it was not a security concern however in Vista they fixed this "non-issue" so does that mean it was really an issue?

At the time I created the paper the information was too sensitive to broadcast but now that Vista handily keeps you from changing system time via programs I think it's time to let people try this out on their old system just for fun to see if it works on any new programs.

The code and exe sample for the clock forwarding is here: http://www.trustsecurityconsulting.com/Downloads.html

There is a program to test the clock/certificate issue and another (if you are affected by this problem) to watch for and correct wild clock changes that cause the issues mentioned in the paper.

Enjoy!
David

Using Neural Nets to Compare Ultrasound Data

Our company had done some volunteer work for a company that needed to match ultrasound "image" data a while ago. The only sample data they had was a picture of the ultrasonic wave signature graphed on the computer. With a little ingenuity we were able to screen-grab the picture and split the sample into segments for a training data set and a testing data set. While using picture data of sound wave forms in a neural net is possible it's not recommended. However we were able to match at 99% confidence with 3 subjects.

Given unlimited training time and a genetic learning algorithm applied to the output a net can "find" variations in the data and learn which ones are significant and which are not. With more up-front thinking and some math magic you can get your data into a fast-training layout with better results.

AI in DB Record Matching (MPI)

What is Fuzzy Logic?
Fuzzy logic is a term that describes the concept of "maybe" where things aren't always yes, no or 1 or 0.  This third alternative of maybe is the area in which humans operate on a daily basis.   We look to computers all the time for definitive information and we either find what we're looking for or we don't. What fuzzy logic does is allow for the third alternative; that just because something is not a definite Yes doesn't mean it's a No. With most 5 gl languages now you have the ability to define custom variable types. Pascal has had that ability for 30 years making Pascal a language before it's time. With custom defined types you can return results to calling functions in a natural readable way that makes sense in the context of fuzzy logic.
I finished integrating a fuzzy logic parser that I wrote some years ago. All that remained was to put the cut-off thresholds in the main program and call the algorithms and see if they were up to the job. It took a while to get the thresholds fine-tuned for the different types of data but the algorithms are sound. With fuzzy logic parsing you can determine what type of data is contained in a field (like phone number, SSN, DL Number or birth date) and you can compare two values and get a confidence level back. By adjusting the acceptable confidence window you can fine tune your data matching.

I was able to quickly build a database record analyser to match data in different tables and to find duplicate records. The results are uncanny and the more data elements you throw into the mix the better it does. It nailed name misspellings, date transpositions, addresses that were written differently like 7th street -vs- Seventh St. etc. It even hits on similar sounding names very close to the performance of soundex but without the high number of false positives.

It has been really fun to work on some of these projects that I designed years ago but never got the chance to implement. A little vision goes a long way.

How To Recover From Bad Update or Delete Querries in SQL Server 2005

It happened to me... the good old update query without the where statement :( The end result was zeroing out hand-coded data that took me about 4 days to enter.

After searching the web for a while I came up empty. Finally I thought of a different way to run the query and found another blog that mentioned 3 products. The only one that worked with SQL 2005 was "SQL Log Explorer". I tried the eval version and after a couple tries got it to load the 54GB log from my DB that was 104 gigabytes. Needless to say this took a while.

Once I narrowed log entries to the appropriate table and date range I found one of the entries from the offending query. All I needed to do at that point was right-click the entry and tell it to "undo". What resulted was the generation of about 50 pages of SQL statements that I fed into SQL Manager and voila!!!! All my data was back.

SQL Log Explorer is an impressive product! I highly recommend it.

Also of note: SLE does not allow you to work on any db in the trial version... it only lets' you work on their DB and on the "pubs" db. As for me, I wasn't tied to my db name so I renamed my DB to "pubs" and ran the program on it.

We have now placed an order for SLE just for those special times when one of us shows our human side.