JoinUO.com

UO Demo • Connecting with client 1.25.35

2011-01-16T23:44:00+00:00

We all know that we can multi-client with the Ultima Online Demo from 1998 using UoDemo+ http://uodemo.joinuo.com/index.php?title=Uodemo%2B and UoDmClnt http://uodemo.joinuo.com/?title=UoDmClnt, both patched versions of the original uodemo.exe. As an alternative to connecting to the UoDemo+ server, I've re-investigated the possibility to connect with a real client. I'm saying re-investigated, because more than a year ago Derrick and I were already testing with client 1.25.35. We then discovered that chatting didn't work and the client would hang after certain time. The problems we encountered back then were caused by UO protocol changes.

Now, using the UoDemoDLL technique http://www.joinuo.com/forums/viewtopic.php?f=32&t=608 I've developed a Packets.DLL which you can attach to the demo. This DLL will intercept the packets it receives from a client and then patches those packets to make them compatible. For now, this DLL will only work with client 1.25.35 (possibly others too). Protocol changes are a common thing in the lifetime of UO and cause a lot of pain in the free shard world.

For client 1.25.35 I coded two actions:

To demonstrate how it works and that it works, I've made a test video which you can download/view here: http://download.joinuo.com/video/UoDemo%20Video;%20Connecting%20with%20a%20real%20client,%20working%20books%20and%20no%20more%20Z%20problem.wmv

Client 1.25.35 shows that books are actually writeable in the demo, the message you get when you try to write in a book is a demo client limitation and not a server limitation!

The Z-problem described originally in this post : http://forum.joinuo.com/viewtopic.php?f=32&t=509 does not exist with client 1.25.35. So whatever is causing it in the original demo, is a client issue and not a server issue!

The attached ZIP contains a patched client and the DLL. To make it work I had to patch out the client encryption.

Statistics: Posted by Batlin — Sun Jan 16, 2011 11:44 pm

UO Demo • Re: Let's talk about Karma & Fame

2011-01-16T18:50:09+00:00

I checked it out, and I find it fascinating. I presume that all leftovers are, for the most part, left in for compatibility's sake, and not due to programming laziness. However, I'm of the opinion that there's been many a programmer working on the client, and perhaps more than one of these fellows were inept in the arts of code documentation. C'est la vie.

I always had a soft spot for the Notoriety system. I like to keep things simple.

-Bicchus Dicchus

Statistics: Posted by Bicchus Dicchus — Sun Jan 16, 2011 6:50 pm

UO Demo • Re: Let's talk about Karma & Fame

2011-01-14T23:56:18+00:00

Hmm I find it odd that the T2A demo, which came out before UOR, has functions for fame and karma.
These two variables didn't exist in the Notoriety system, which was the standard of the day.

The demo contains leftovers from the Notoriety system but it's not active inside the EXE. If you have time, check out the scripts and read them with a correct analysis of the getCompileFlag function : http://uodemo.joinuo.com/index.php?title=GetCompileFlag

Inside the EXE there's also a reference to a "remnoto" script which in my opinion stands for "remove notoriety". The script itself is not included in the DAT file.

Another interesting script is the "repconvert" (reputation convert) script which actually converts notoriety to karma/fame. Pretty cool stuff in there.

Statistics: Posted by Batlin — Fri Jan 14, 2011 11:56 pm

UO Demo • Re: Let's talk about Karma & Fame

2011-01-14T22:58:36+00:00

Hmm I find it odd that the T2A demo, which came out before UOR, has functions for fame and karma.
These two variables didn't exist in the Notoriety system, which was the standard of the day.

I reckon they had been considering and working on the Reputation system for a while before it was introduced.

Always get nostalgic for the old Noto days though.

-Bicchus Dicchus

Statistics: Posted by Bicchus Dicchus — Fri Jan 14, 2011 10:58 pm

UO Demo • a bug in the server's packet receiving code...

2011-01-12T14:17:00+00:00

While preparing my next project I discovered a flaw in the server's code that could lead into the server dropping certain fragmented packets.

Let me first explain the server's loop per receive:

Handling the packets consist of two big parts:

UO packets consist of two types of packets, packets with a fixed length and packets with a dynamic length. When handling packets with a fixed length both the server and the client must speak the same language. If one of them send a packet of different length other than the expected length, to other side will turn into an endless loop waiting for the remainder of the packet to be received. If you send too much data, then the other side will process the excess as if it were a new, seperate packet.

The bug is this: when the server checks for the packet to be valid, it requires a packet ID. Therefor the first check the server does is ensuring atleast one byte has been received. After obtaining the packet ID it will query its internal database to know if the packet is of fixed length or dynamic length. In case of fixed length, the server will not continue handling the packet until at least all data has been received for it. In case of dynamic length, the server will check the next 2 bytes in the packet as they contain the size of the dynamic data. However, the server code DOES NOT test that atleast 3 bytes have been received and that could be a problem.

The following python program demonstrates the potional problem:

Code:: import socket import time s = socket.socket() s.connect(("192.168.192.112", 10016)) packet = "\x03" s.send(packet) time.sleep(1) packet = "\xA4".ljust(149,"\xFF") s.send(packet) time.sleep(3) packet = "\x03" s.send(packet) time.sleep(1) packet = "\x77" s.send(packet)

First packet 0x03 is send, this is a dynamic packet.
The server's buffer will contain 0x03 00 00 00 00 and so-on.
Therefor the dynamic-length checked will be 0x0000.
The packet is discarded.

Next packet 0xA4 is send, this is a fixed packet that is ignored by the server.
Normally this packet contains system information about the client's PC. I filled it here with 0xFF.
Therefor the buffer now contains 0xA4 FF FF FF and so-on

Then packet 0x03 is send.
The buffer now contains 0x03 FF FF FF and so-on.
The dynamic-size is now 0xFFFF, and the server expects to receive 65535 more bytes and will wait.
Remember, there is only one valid client byte in the buffer.

We send 0x77.
The buffer now contains 0x03 0x77 FF FF and so-on.
There are 2 valid bytes from the client in buffer now, 0x03 and 0x077, the first FF is still garbage which the server is interpreting as it were client data.

I'm again gonna try to explain the effects of the bug:
If the packet arrives fragmented and in-case of a dynamic lengtht packet, less than 3 bytes are received, then the buffer will contain an invalid value. As the server fails to check that it has atleast 3 bytes, this will be a problem. The packet may be dropped, or may be parsed incorrectly or if you're lucky, the server keeps waiting up to the next byte. What will happen is up-to the packet being received, the new length and the data already in the buffer! In worst-case, the dynamic-length data is gonna be interpreted as if it were a seperate packet causing the client to hang. Validate packet siz.pngThe red line is where the dynamic-length is obtained.

Statistics: Posted by Batlin — Wed Jan 12, 2011 2:17 pm

UO Demo • Re: The Multi Script Bug - Ships & Houses

2011-01-10T11:51:35+00:00

Haha! Great find and very nice research with the CVs!

Hilarious.

Thanks for clearing this up. It definitely gives us an indication regarding the caliber of future bugs we'll be facing in the demo

Statistics: Posted by roguan — Mon Jan 10, 2011 11:51 am

UO Demo • The Multi Script Bug - Ships & Houses

2011-01-10T11:34:04+00:00

First I encourage you to read roguan's post about ships (if you haven't read it already) : http://www.joinuo.com/forums/viewtopic.php?f=32&t=615.

I've researched the source of the "NAME INVALID" issue roguan discovered using Process Monitor.
If you look at his screenshot (see his post), you will see that the name looks valid and that there is no real visible reason for the function to fail.

However, if we investigate the actual loading and parsing of multi.txt.q using a debugger, then we see that the script name ends with an invalid 0D character (return character or \r).The Multi-Bug.png

Why is that 0D character there? If you look at the screenshot you may notice that OSI's code is scanning for a 0A character (newline character or \n). The 0D 0A sequence as a line seperator is common in the DOS/Windows world. The 0A sequence is common in the Unix/Linux world. http://en.wikipedia.org/wiki/Newline

Let's look at the multi.txt.q itself (using a hex editor so we can see the line seperators): multi.txt.q.png

Why is the demo scanning for a Unix-sequence whereas the file itself is Windows-based?

It's obvious that OSI's server code ran on Unix systems, check out this CV:
http://www.talisman.org/~erlkonig/resume/

Unix Systems Analyst (June 1994 - April 1997)
Origin Systems

Sole (until mid 1996) Unix systems administrator for a medium-sized network of Silicon Graphics machines at Origin Systems, ranging from desktop workstations to a half-million dollar SGI Onyx RE2, working tangent to the PC/Macintosh technical staff. Involved in or responsible for all Origin Internet services. My game credits include:

ULTIMA ONLINE: Shattered Legacy pre-alpha

Also check out this one: http://jasons.wumple.com/Resume/

Senior Software Engineer, Origin Systems, studio of Electronic Arts, 11/1996 to 12/1997
Released Ultima Online, the award winning massively multiplayer online game.
Programmer, Ultima Online, an award winning massively multi player online role playing game supporting tens of thousands of simultaneous players.
Designed and implemented auto patching system (client and server), interprocess script communication system, multiobj system, boats, player housing, global hint system, game master tools, object decay, communication crystals, magic item creation system. Worked on most systems in the game.
Extensive use of C++, templates, STL, TCP/IP (sockets), Win32, MFC, Win32 threads and POSIX pthreads with development under Linux, Solaris, and Windows.
Lead Programmer from 1/1998. Maintained task lists, reported progress, managed small team of engineers.
On call 24/7. Fixed many critical problems whenever they occurred.
Made numerous CPU and memory optimizations which greatly improved performance of game servers. Improved stability of service.

My guess is that they converted the file itself to a Windows based file, but didn't do effort to fix/test the code when they converted the server code from Unix to Windows for creating the demo. Also, the Time Bug I described here : http://www.joinuo.com/forums/viewtopic.php?f=32&t=569, is a bug I believe to be of the same origin, a unix-to-windows conversion related issue.

Now, how to fix this bug?

1) You can use roguan's approach which is quite technical (see his post).

2) You can convert the multi.txt.q file back to Unix format by replacing the 0D 0A sequence by a 0A sequence using a hex editor.

3) You patch the EXE, again using a hex editor, so that the demo scans up-to-the 0D character instead of the 0A character.
The Fix.pngAt 0x21B00E change 0x0A in 0x0D. That's it!

Note, an-easy-to-use-and-free hex editor is HxD : http://mh-nexus.de/en/hxd/
I've also attached a pre-patched multi.txt.q file to this post.

Statistics: Posted by Batlin — Mon Jan 10, 2011 11:34 am

UO Demo • Re: NPC states - The Old and The New

2010-12-31T18:21:54+00:00

Nice thanks Batlin, and OSi
Hapy New Year!

Statistics: Posted by Derrick — Fri Dec 31, 2010 6:21 pm

UO Demo • NPC states - The Old and The New

2010-12-31T09:10:24+00:00

I was working on decoding and understanding more of the NPC AI and I suddenly stumbled upon an uncalled function inside the EXE. I documented the function and named it "FUNC_GetNPCstateString". This is a screenshot of the function: FUNC_GetNPCstateString.pngThis is the C version of the same function:

Code:: const char *GetNPCstateString(int NPCstate) { const char *result; switch(NPCstate) { case 0x00: result = "Seek Food"; break; case 0x01: result = "Seek Shelter"; break; case 0x02: result = "Purse Shelter"; break; case 0x03: result = "Seek Desires"; break; case 0x04: result = "Purse Desires"; break; case 0x05: result = "Eat Food"; break; case 0x06: result = "Loiter"; break; case 0x07: result = "Runaway"; break; case 0x08: result = "Talking"; break; case 0x09: result = "Attack Target"; break; case 0x0A: result = "Idle"; break; case 0x0B: result = "Wander"; break; case 0x0C: result = "Sleep"; break; case 0x0D: result = "Following"; break; default: result = "Unknown State"; } return result; }

In the past I had already found an unused function that would SuperBark the current state of the NPC. However, that function did not match the actual states being used inside the core and scripts. Here's a screenshot of that function: FUNC_SuperBarkNPCstate__OLD.pngAgain, the C version:

Code:: void FUNC_SuperBarkNPCstate__OLD(NPC *NPCobject) { const char *result; char TempStringBuffer[512]; switch(NPCobject->CurrentState) { case 0x00: result = "Wander"; break; case 0x01: result = "Pursue"; break; case 0x02: result = "Runaway"; break; case 0x03: result = "Combat"; break; case 0x04: result = "Following"; break; case 0x05: result = "Talking"; break; case 0x06: result = "Loiter"; break; case 0x07: result = "Sleep"; break; case 0x0A: result = "Idle"; break; default: result = "Bad State"; } // Format the state into nice output sprintf(TempStringBuffer, "myState: %d (%s)", NPCobject->CurrentState, result); // Bark the nicely formatted state NPCobject->SuperBark(TempStringBuffer, -1, -1, -1); }

If you look closely you will see I added the tag __OLD to the function because it dates most probably from an older period in OSI's UO development.

How did I know which state function is the correct one? Well, the biggest clue is the sleep-state. In the first function I posted that equals state 0x0C, in the other one it is 0x07. There is a script command "goSleep". Let's take a look at that one: COMMAND_GoSleep.pngFor most people, the C version:

Code:: int COMMAND_goSleep(int NPCserial, int SleepTimeInTicks, int PostSleepState) { int result; // Validate the PostSleepState if(PostSleepState >= 0 && PostSleepState < 14) { // Ensure the given NPC is really a NPC object NPC NPCobject = ConvertObjectBySerialToNPCobject(NPCserial, "setNPCState"); if(NPCobject != NULL) { // Go set the actual sleep state NPCobject->GoSleep(SleepTimeInTicks, PostSleepState); result = 1; } else result = 0; } else result = 0; return result; }

The most important thing we learn from this function is the range check of the PostSleepState. This range equals the switch/case-statement in the top function. Next, let's look at the actual NPC class function GoSleep: FUNC_NPCobject_GoSleep.pngAt 0x004AB8D6 you see "push 0x0C", this is the NewState being set when the NPC goes into sleep state. Perfect match with that first function.

Thank you OSI for leaving those 2 functions in there, it's gonna be a whole lot easier to understand the NPC AI now and it gives a glimp from your old AI. More to come, but that's going to be for the new year, old year is today. Happy holidays to you all!

Statistics: Posted by Batlin — Fri Dec 31, 2010 9:10 am

UO Demo • CanSeeLoc aka OSI's LOS (Line Of Sight)

2010-12-30T11:26:29+00:00

This article is about the LOS algorithm which we can find inside the demo. Everything starts with the 2 script functions we have: canSeeLoc and canSeeObj. Both of them will end up calling a class function I named CanSeeLoc located at EIP 0x46ADA5. FUNC_XXX_CanSeeLoc is the internal name I gave (because it belongs to a class I could give a proper name for yet)

Let's first take a look at canSeeLoc: canSeeLoc.pngcanSeeLoc takes an object as parameter and a location variable. The function will thus test if a certain object can see a certain location. The object is converted to a location by taking its location and adding halve the height of the object to the Z-axis. Not shown in the dissambly but all mobiles have a fixed height of 16 (even those tiny, nasty rats).

canSeeObj is a bit different: canSeeObj.pngIt takes two objects as a parameter and will check if the target object is a hidden mobile. If the target object is a hidden mobile no further action is taken and 0 is returned. Also, looking at 'yourself' will result in a quick termination of the function with a return value of 1. Otherwise the canSeeObj class function of the item class is called. Refer to this post about the class structure for the game objects : http://www.joinuo.com/forums/viewtopic.php?f=32&t=539.

This is the canSeeObj of the item class: ItemObject_CanSeeObj.pngAgain, both objects are converted by adding halve their height to the Z axis and then calling the internal CanSeeLoc function.

The actual magic is going on in the CanSeeLoc (FUNC_XXX_CanSeeLoc) function. This is a screenshot of all cross references to this function :xrefs to CanSeeLoc.png

I converted all the assembler to readable C(++) code and turned it into a DLL using the UODEMODLL technique introduced here : http://www.joinuo.com/forums/viewtopic.php?f=32&t=608. You can download the full source code and a precompiled DLL here : http://download.joinuo.com/UoDemoDLL/Version%201.1.rar. Posting the full source code would only clutter this post and if you're interested I guess you're gonna download it anyways .

This DLL contains advanced hacking techniques that redirect all calls to CanSeeLoc (see the picture above) to the DLL, tthe DLL will then call both the original function and the decompiled function. I did this so we could/can detect mismatches (= errors in the decompilation).

One note about the algo, there are some "<< 16" lines in there, they act as an optimization to avoid slow floating point math in the algo. If someone else also understands the optimization and can explain it better than myself, please do so.

Screenshot of the DLL in action: CanSeeLoc DLL.png

As always, feel free to ask any questions.

Statistics: Posted by Batlin — Thu Dec 30, 2010 11:26 am