Reverse Engineering of Xbox Security Method 3

Once I saw Brandon Wilson’s note on Xbox 360 Controller Security and become curious about how its really works and decided to take a look by myself.

It was my first Xbox 360 related work and Xbox 360 seemed for me much more simple from software and security point of view than PS3. Just a few modules, kernel (xboxkrnl.exe) and 256kb hypervisor which is used mostly for kv (key vault) management.

At PS3 there are about 400 modules (which are sharing alot of the same and unused code), big hypervisor, kernel and SPUs.

Microsoft ships XDKs (Xbox Development Kit) with zeroed root encryption key, and so updates provided with it are not encrypted and contains debug symbols, even for kernel. And kernel itself looks like a lib and can be debugged on XDK? (Not sure about that, never had one)

So it was a matter of seconds to find XSM3 algo.

After Xbox 360 gets all descriptors from controller, and interface descriptor string “Xbox Security Method 3, Version 1.00, 2005 Microsoft Corporation. All rights reserved.” it starts to send auth packets.

There are my sniffed packets:

UsbdSecXSM3GetIdentificationProtocolData

49 4B 00 00 17 04 E1 11 54 15 ED 88 55 21 01 33 00 00 80 02 5E 04 8E 02 03 00 01 01 C1

UsbdSecXSM3SetChallengeProtocolData

09 40 00 00 1C DE EB 91 87 66 B0 E3 C0 B2 6C 05 6D C8 67 E2 E7 D6 A5 DC 71 6F 21 1F B4 32 28 A0 C2 89

UsbdSecXSM3GetResponseChallengeProtocolData

49 4C 00 00 28 7D 66 8F 48 76 EC EB E9 61 B7 81 82 08 84 B2 7E 73 1F 7A 92 47 83 F8 5E 78 22 11 B9 9B FC AD D6 F4 D9 1E 04 7E C1 C0 0B 9A

UsbdSecXSM3SetVerifyProtocolData1

09 41 00 00 10 60 17 AC 73 E5 0F 10 D4 10 F5 C1 B5 8F 63 56 9B 36

UsbdSecXSM3GetResponseVerifyProtocolData1

49 4C 00 00 10 60 7E 0C 62 AE 46 94 E7 14 BA 3D 70 33 7E 93 DF 09

UsbdSecXSM3SetVerifyProtocolData2

09 41 00 00 10 47 A2 4C A3 97 DF AC CC 29 48 F1 D0 08 11 D1 FD 57

UsbdSecXSM3GetResponseVerifyProtocolData2

49 4C 00 00 10 49 3A E1 F9 8C 91 73 C3 FA A2 07 D3 CC 1E 2F 17 A0

Data format:

• [5 bytes - cmd header]
• [data]
• [1 byte - chksum (xor of data bytes)]

UsbdSecXSM3GetIdentificationProtocolData contains Serial, Category, VendorID and other data.

Category and VendorID are important bytes. Each device type like wireless controller, wired controller, webcam, memory unit, etc. has their own category. VendorID can be 2 types: Microsoft / 3rd party.

Depending on this bytes encryption key is chosen.

The full reverse engineered algo among with my sniffed usb traffic can be found on my github.

Algo is big, obfuscated and unfortunately contains custom crypto algorithms. I tried to cheat on it: checked pc drivers, xbox sdk, windows debug symbols with hope that Microsoft used it somewhere else, but nothing of that succeeded so I ended up rewriting this algo from PowerPC asm to C.

I did this research back in 2013… who would guess back then that Hex-Rays will release PowerPC decompiler ???

The problem with it that Xbox 360 encrypts hardware serial with fixed key and sends it to usb device. From now on Xbox uses per-console key from kv (key vault) for crypto.

This keys are precalculated from hardware serial on factory, and secret part to do serial to per-console key is put to authorized devices.

Okay, so what now?

Xbox controller, Skylanders Portal etc. uses Infineon TPM chip. But what about unauthorized devices? There are so many of them! Devices like Cronus and XIM need wired controller to bypass auth. Looked at some Chinese ones but they use real MS chips. Seems Datel was the only ones who really cracked it and were selling their own controllers. In one of them I found out 2 MCU’s. One is SiTel SC14450 - a DECT phone controller, probably abused for the radio interface. The other is an unknown 16 pin microcontroller, marked ‘Raw Science’ (this is a Datel company).

DFU firmware for SC14450 can be found in updater, it is CompactRISC CR16C 16-bit microcontroller, but firmware does not contain anything interesting, all cmds are passed direct to the Datel ‘Raw Science’ chip.

Datel chip was decapped. Looks like a smart card or SIM chip, 6 wires, rom, flash & ram. But I was not able to recognize this chip. Wild guess was that MCT = MicroChip Technology, but its not PIC.

Masked rom is not big and easily recognizable so it was not hard to get bootrom dump. Its based on 8051 architecture, but it did not hold anything interesting or something that will help us acquiring secret part.

So at this moment Xbox Security Method 3 is reverse engineered but secret algo to do serial to per-console key is still secret.

To be continued in Part 2…

If you liked this post, you can share it with your followers or follow me on Twitter !