Posts Tagged Xbox

Python script to generate Xbox save signing keys

A while ago I discussed how save signing for the original Xbox works and the methodology to generate the correct key to resign a save.  Save Hacking is becoming something of a legacy these days and I’m keen to do what I can to enable others to continue to work on what I guess is called ‘retro’ these days.

With that in mind I’ve uploaded a quick and dirty Python script to generate Xbox save signing keys to GitHub.

Using xbox-save-sig is as simple as passing in the path to a default.xbe file:

python xboxsig.py -i /path/to/default.xbe

The output will be a text representation of the digital signature for save signing:

737E064C0236BA3E7140EC2B91D76766

The script can output the key in a selection of formats and being Python can run on many different systems.

The script is licensed under the MIT License, which is pretty permissive, so feel free to fork, improve and put it to good use!

Tags: , ,

Xbox save resigning – a technical overview

This article covers the saves signing process used by the original Xbox.  Only the standard signing process is discussed, the “noroam” signatures are not covered.  All code is based on my own work or derivative work of others.  The language used in examples is Delphi and assumes a familiarity with programming concepts such as the use of records or structures for reading in data.

The reason I’ve never previously discussed the Xbox save signing procedure in public is concern that by doing so would negatively impact Xbox Live.  Now that the original Xbox and it’s games are unable to access the Xbox Live service the following information is relatively harmless.

A brief history

The original Xbox used digital signatures to validate not only executables it was loading but save data, with this it was impossible to change or tamper with a save without updating the file’s digital signature using the correct key.  This key was generated using data contained within the default.xbe and a key used by the Xbox Operating System.  The save data was then run through a SHA1 HMAC routine and the 20 byte result appended to the save data to confirm integrity. This result, or digest, was usually found at the beginning or the end of a file although it can potentially be located anywhere in the file if the developer was feeling adventurous.

Obtaining the XBE key

Each Xbox game has a default.xbe file, this is the executable that is loaded whenever you start a game. It contains various information including the title ID, age rating, game region and the all important signature key.  Don’t get confused, this is not the key used to sign the saves but is an essential piece in the process.
The first step is to obtain the key directly from the XBE. This is stored in a certificate area of the file, the address of which is located in the XBE header. Before we can retrieve the key we need to define the structure of the XBE Header and the Certificate data we will be retrieving:

?View Code DELPHI
// XBE sections
unit xbestruct;
 
interface
 
type
ByteArray = array[0..15] of byte;
 
type
TxbeHeader = record
	m_magic : cardinal;                    // magic number [should be "XBEH"]
	m_digsig : array[0..255] of char;      // digital signature
	m_base: cardinal;                      // base address
	m_sizeof_headers : cardinal;           // size of headers
	m_sizeof_image : cardinal;             // size of image
	m_sizeof_image_header : cardinal;      // size of image header
	m_timedate : cardinal;                 // timedate stamp
	m_certificate_addr : cardinal;         // certificate address
	m_sections : cardinal;                 // number of sections
	m_section_headers_addr : cardinal;     // section headers address
	m_init_flags : cardinal;
	m_entry : cardinal;                         // entry point address
	m_tls_addr : cardinal;                       // thread local storage directory address
	m_pe_stack_commit  : cardinal;                // size of stack commit
	m_pe_heap_reserve : cardinal;               // size of heap reserve
	m_pe_heap_commit  : cardinal;                 // size of heap commit
	m_pe_base_addr : cardinal;                   // original base address
	m_pe_sizeof_image : cardinal;               // size of original image
	m_pe_checksum : cardinal;                // original checksum
	m_pe_timedate : cardinal;                   // original timedate stamp
	m_debug_pathname_addr  : cardinal;            // debug pathname address
	m_debug_filename_addr : cardinal;           // debug filename address
	m_debug_unicode_filename_addr : cardinal;   // debug unicode filename address
	m_kernel_image_thunk_addr : cardinal;        // kernel image thunk address
	m_nonkernel_import_dir_addr : cardinal;      // non kernel import directory address
	m_library_versions : cardinal;               // number of library versions
	m_library_versions_addr : cardinal;          // library versions address
	m_kernel_library_version_addr : cardinal;    // kernel library version address
	m_xapi_library_version_addr : cardinal;      // xapi library version address
	m_logo_bitmap_addr : cardinal;               // logo bitmap address
	m_logo_bitmap_size : cardinal;               // logo bitmap size
end;
 
pTXbeHeader = ^TXbeHeader;
 
type
TxbeCertificate  = record
	m_size : cardinal;                          // size of certificate
	m_timedate : cardinal;                     // timedate stamp
	m_titleid : cardinal;                       // title id
	m_title_name : array[0..63] of widechar;                // title name (unicode)
	m_alt_title_id : byteArray;            // alternate title ids
	m_allowed_media  : cardinal;                 // allowed media types
	m_game_region  : cardinal;               // game region
	m_game_ratings  : cardinal;                  // game ratings
	m_disk_number : cardinal;                   // disk number
	m_version : cardinal;                       // version
	m_lan_key : byteArray;                 // lan key
	m_sig_key : byteArray;                  // signature key
	m_title_alt_sig_key : array[0..15, 0..15] of byte;     // alternate signature keys
end;
 
pTxbeCertificate = ^ TxbeCertificate;
 
implementation
 
end.

The TxbeHeader record contains a lot of data but the two most important sections are m_certificate_addr, and m_base.  These two values give us the address of the certificate.

The following code retrieves the key from an XBE file:

?View Code DELPHI
var
    MS : TMemoryStream;
    xbeHeader : pTxbeHeader;
    xbeCert : pTxbeCertificate;
begin
 
    MS := TmemoryStream.Create;
    MS.LoadFromFile(FileName);
    new(xbeHeader);
    new(xbeCert);
    //Read in header and certificate
    MS.Read(xbeHeader^, sizeof(xbeHeader^));
    MS.Position := xbeHeader^.m_certificate_addr - xbeHeader^.m_base;
    MS.Read(xbeCert^, sizeof(xbeCert^));

We can now access the sig_key directly

?View Code DELPHI
xbeCert^.m_sig_key

Generating the signing key

The key we retrieved from the XBE is not used to directly sign save data, instead it is used in a SHA1 HMAC with the Xbox key to produce the actual key we need. Thankfully we don’t need to determine the Xbox key every time we need to sign or verify a save as it is a constant.

The following is a textual representation of this key, to use it you must first convert it to a 16 byte array:

5C0733AE0401F7E8BA7993FDCD2F1FE0

Once you have this key in a byte array simply run both the XBE key and the Xbox key through a SHA1 HMAC (referring to the documentation for your SHA1 HMAC function as to whether you need to pass data or memory addresses to the function). The output should be a 160bit digest truncated to 16 bytes (the last 4 bytes are not required and should not be used or present).

?View Code DELPHI
var
    digest : T160BitDigest;
begin
    digest := CalcHMAC_SHA1(addr(xboxKey), 16, xbeCert^.m_sig_key, 16);

Determining the data to process

As noted before the digital signature is usually found before or after the actual save data.  It is entirely possible to store this signature at any location in the file and treat this location as all 0’s during the HMAC process.  Since we cannot rely on all developers using the same location for the signature we must determine the location ourselves.

The fastest and easiest way to do this is to start a new game and make a save at the earliest opportunity.  If the game saves any options you change this is ideal, otherwise start playing and save as soon as you can.  Copy this save to you PC and label is SAVE A.

Load the game again and create another save with as small a difference as you can but ensuring something is different.  As before, if the game saves option changes you should change only a single item and re-save. Copy this save to your PC and label it SAVE B.

Open both saves in a hex editor and visually compare the two, with little differences between them you should easily spot the 20 byte digital signature either at the start or the end of the file.  You can use an automatic file comparison if your editor supports it but in my experience I can find the sig faster by eye.  The signature in both files should be wildly different with the actual save data very consistent, aside from the minor differences you saved earlier.

Once you have found the sig location you should exclude this from the HMAC routines.  If the sig is at the end of the file you should HMAC all data up to the last 20 bytes, if it is at the beginning then all data after the first 20 bytes should be processed.  If the signature is located at another location in the file then you will need to experiment as to what data to HMAC.  A common trick when generating checksum’s is to treat the area containing the result of the checksum as all 0’s during the processing stage and write the result back to this location once complete.  Please note, it is rare to see this, most saves store their signature at the beginning or end of the file.

Generating the signature

Now you have the correct key and the data to process you can generate or check signatures for that particular games save files.  Similar to how the key was generated, the actual signature generation is a SHA1 HMAC of the save data (excluding the existing signature) with the key.

In the following example the save signature is located at the end of the file and the data is copied to a different memory stream for processing. The previously generated key is stored in a 16 byte array named sigKey

?View Code DELPHI
mem := TMemoryStream.Create;
mem2 := TMemoryStream.Create;
mem.LoadFromFile(FileName);
mem.Position := 0;
mem2.CopyFrom(mem, mem.Size - 20);
digest := CalcHMAC_SHA1(addr(sigKey), 16, mem2.memory^, mem2.size);

Before making any changes to a save you should confirm that the signature you generate matches that already present on an unaltered save.

It’s worth bearing in mind that a lot of saves contain a checksum as well as a digital signature.  The most common being a CRC32 or a simple addition of bytes, you must recalculate this before the digital signature should you make any changes to the file.

Conclusion

The process of creating a digital signature for Xbox saves is fairly simple and can be summarized as

  • SHA1 HMAC “XBE key” using “Xbox key”
  • Truncate resulting 160 bit digest to 16 bytes to create the “signature key”
  • SHA1 HMAC Save data using “signature key”
  • Compare resulting 160 bit digest to existing signature or write back to save file.

If you use the information above to create anything or produce a unique save, drop me a line in the comments, sometimes the most interesting time in a games life is long after it was released..

Tags: , , ,

GTA: SA Censor Remover Online AKA Conversion Hell

I often get requests for versions of my tools that will work on non-Windows operating systems, such as Linux and MacOSX.
Because I use delphi, which is primarily a Windows programming tool, this often isn’t practical or even possible.

A few months ago I set about thinking of ways I could provide the same tools and abilities to those who don’t run Windows or have restrictions set on what they can download. I eventually decided to recreate one of my more popular tools, GTA: SA Censor Remover, in an online edition.

This wasn’t as easy a task as I had first assumed, my PHP skills were a little rusty from years of neglect in favour of Windows application development and PHP is an inherently poor language for working with binary files. I could have deviated from my original intention and written the online version in Perl, but I’m not that much of a masochist and I’m much more familiar with PHP.
After a few hours I was able to knock up a basic version that worked fine with PS2 and PC saves, even with PHP’s poor file handling routines, but came against a huge stumbling block with the Xbox saves.

Xbox saves are protected by a digital signature, change even one byte and the signature is invalidated and the save will not load. Of course I know how to fix this signature but this is where PHP let me down immensly.

PHP is a loosely type language, this means it’s great for beginners but sometimes you need to be able to control the language and it’s variables precisely, which is exactly what I needed to do. For reference, Delphi is a strongly typed language and is much better for it in my opinion. I can also access raw memory directly with Delphi which makes creating digital signatures a breeze.

I spent hours trying different ways of representing the binary data I needed to fix the digital signature in the save, I trawled the PHP manual for hints and comments. I even asked an experienced PHP programmer who writes scripts for a living daily and he was unable to help me or offer anything I hadn’t tried already.
Eventually I gave up, I’d chosen the wrong tool for the job and had paid bitterly for it.

Last night I was intrigued to see that the makers of Delphi, Codegear, are planning a “Delphi for PHP” and this rekindled my interest in PHP. Looking for a project to start I remembered about the work I had done on GTA: SA Censor Remover Online and how I had left it to rot on my dev machine.
I decided that I couldn’t let this go to waste and was a perfect challenge to get my PHP skills back on track, after all if I couldn’t find the answer from the manual and experienced users of PHP than I’d have to find it myself.
Eventually after a few tests, all producing the wrong digital signature, I commented out all the existing code and returned to the basic task: Create a binary representation of a series of bytes and pipe this along with the raw file data to the required function.

It was a matter of minutes before I had a working function. I spent a few hours tweaking the code and making sure it worked properly, again having to fight PHP’s poor file handling tooth and nail, until the final product was ready for release.

It’s been a hellish struggle and at times I wondered if the task was even possible given the poor performance of PHP in this area and lack of information out there, even from PHP experts. I won’t be retro-fitting any of my other tools at this time but if the need arises, or another prgram is created that would bendfit from an online version, I’ve got a solid groundwork to enable access to my software to everyone, regardless of what operating system they use.

Tags: , , , , , ,

Fragmented usernames, gamertags and profiles

I love online gaming.

I don’t get enough time to participate these days, nor do I own any of the “next gen” consoles, which is where I am led to believe the most enjoyable online experience can be found, particularly if you are like me and use your PC for work and your consoles for fun.

Earlier this week Sony opened up a service that allowed you to pre-book your online username/gamertag/persona (call it what you will) if the Playstation 3 hasn’t reached your country yet or you just don’t fancy paying the eBay prices for their latest toy.
I’m all for these services and decided to see if such a service was offered by Xbox Live. Apparently it is, so I duly entered my details only to find that my gamertag was already taken…

I’ve been using the online persona gothi since the 90’s, I’m pretty fortunate in that it’s an usual word (although at the time I was not aware of it’s religious connotations). I’ve made a name for myself by posting, and administrating, on many forums, creating websites, I’ve released hundreds of Xbox & PS2 saves, played in online tournaments, beta tested online PS2 games, written several widely used software programs and it’s even got to the point where some people think entering my username into a game makes you invincible. It doesn’t by the way, however using the cheat save I made for the Xbox version of XIII, which features my profile name, does.

I like to think I made myself known, I even own gothi.co.uk and the fact I cannot use my username, the one I have used for about a decade, on Xbox Live frustrates me.
I like how Xbox Live operates, one username for all games, but what do you do when your username is taken? Do I have any right to ask that the user of this name gives it up? I appear twice in the top 10 google results for gothi (not bad considering my username is a religous term), does all that give me enough leverage to say that “this is my online persona”?

As it stands both Microsoft’s and now Sony’s approach is reasonable but limited. You choose one name, but it is only unique to that system, and when a new system is released it can be mad scramble to get your name registered. In this case I clearly lost out on Xbox Live. To make matters worse, some games companies, seeing that users want one name across the systems, have created their own private database of usernames that allow you to have one name across their collection of games. Rather than improving matters this makes them worse as now there are even more private areas to register for and more chances to lose possession of your name. Without your name, and the reputation that follows it, who are you?

What I propose is a universal system where users can choose one name for all systems and games, regardless of publisher, designer or manufacturer.
By use of an open API games designers can verify ownership of a gamertag and other information, such as game specific data, can be stored on their own databases.

Is it too late for such a system? Maybe. I truly hope it isn’t and that some order can be brought to the chaos that is forming.

I’d like to know what others think of the fragmentation issue we’re in and how you see it in 5 years time…

Tags: , , , ,

The end of save hacking?

Save hacking is exactly what the name implies, the manipulation of the raw data of the file a game uses to store such things as ammo counts, collected items, unlocked levels and more. It’s been a solid and reliable way to alter many games, be it the number of lives you have all the way up to unlocking removed levels and cheats that the game designers never wanted you to see.

I’ve been interested in save hacking for many years, I was once employed to do this very task, but a worrying trend has appeared recently, particularly in console saves.

Let’s take a (short) trip into the past.

Over the years developers and even the console makers have been finding different ways to make save hacking as difficult as possible. Saves started to use checksums to verify data integrity but these were easily bypassed. Next saves were encrypted by the game, although this too was bypassed by skilled people able to determine the methods used by the game and reverse the procedure. All was still good in the land of save hacking.

Fast forward to 2001 and the advent of the original Xbox console. Saves for this system used a digital signature as standard as well as other methods such as checksums and encryption, altering even one byte rendered the save invalid and the game would not load it.

After some serious hacking of the console and its operating system a method to resign the digital signature on the saves was discovered but the details of how to do this were shrouded in secrecy. A few people, myself included, were able to (re)discover the methods used and create our own tools to assist in save hacking. The most useful of these was a tool that the public could use to resign saves, but only those allowed by those with the knowledge. This secrecy, usually a burden to endeavours like these was a necessary evil; Xbox Live had taken off a storm and, due to the information to resign the saves for Live compatible games being released, the cheaters started to appear. This wasn’t the only way to cheat on Xbox Live but it was one that could be hampered and almost stamped out, so an unofficial and unagreed stance was taken by the few; no-one was to provide information on the resigning process nor provide keys that could affect online play.

Now in 2006 we have the next generation of consoles and save hacking faces new challenges.

The Xbox360 saves contain a digital signature, the methods and keys to resign this signature are still not known, at least publicly. I’ve yet to see hard evidence of a save for this system being hacked, at least in the traditional method.
The PS3, Sony’s latest games console, uses a system based on its work with the PSP.
The virtual PS2 saves it uses, the format of which I have mapped over at PS2 Save Tools, uses a digital signature so again, even changing one byte makes the save useless.
As for the saves generated by PS3 games, these are similar to PSP saves and as such are a totally encrypted file. The only save tools available for the PSP must be run on the PSP to unencrypt the data, at this time it is not possible to unencrypt the PS3 save data on a PC.

Where does this leave us?

I had hoped that the Xbox360 would have been laid bare by now but the DVD drive firmware modification has meant that many of the people with the technical knowledge are focusing on breaking new protections Microsoft put in place. This appears to have slowed down, if not stopped completely, efforts to run unofficial software on the Xbox360, a definite step towards editing and resigning save files.
At this time we are unable to run unsigned code on either the Xbox360 or the PS3 (Linux does not come into this) and in the PS3’s case, this may be exactly what is needed as it is with the PSP.

It’s a grim future for save hacking, the security systems on both the Xbox and the PSP have been bypassed by clever manipulation of certain save files which has resulted in tighter and more complicated methods for protecting the data integrity and ensuring we cannot alter save files.
Will I still be save hacking in 3 years time? I hope so but the future’s very uncertain at this time and not looking good…

Tags: , , , ,

GTA: SA Censor Remover Online

I’ve been doing some work to make the ever so popular GTA: SA Censor Remover available as an online tool, opening it’s abilities to everyone rather than just Windows users.

It’s a cut down version, it lacks many of the advanced features of the full program but does it’s basic task with honours.

It’s currently in development stage, I have the PS2 and PC save support in and working but the Xbox code is giving me some troubles, mainly due to trying to do everything in PHP, definitely a case of the wrong tool for the job.

I’ll post a beta link when I’ve tidied the code up a bit, with any luck I’ll have the Xbox code working by then too.

Tags: , , ,