Wiimmfi logo

Development of Wiimmfi

History

Servers

History: First traffic analysis

2012-11: Analyzing the network traffic of MKWii

In November 2012, Wiimm decided to analyze the network protocol of Mario Kart Wii. The main goal was to detect online cheaters. Another goal was to set up an own server if Nintendo ever shut down its servers, which eventually happened on May 20th, 2014.

Wiimm saved all of his network dumps together with videos of the races for later analysis. This was a good decision and helped a lot when developing Wiimmfi.

2012-12-06: Tool mkw-ana

In December 2012 Wiimm began development of the tool mkw-ana. The main feature was to dump the packets of network dumps (captured by tcpdump or wireshark) in a user-friendly format.

mkw-ana is a hex dumper, that can identify different packet types and split them into different sub-records. Known values are printed in user-friendly formats instead of simple hex numbers.

2013-06-01: mkw-ana prints a live racing table

At the beginning Wiimm was focused analysis of racing data. So mkw-ana was taught to print live statistics on the screen with the current rank and other status values.

History: Wiimmfi development

2014-02-27: Nintendo announced WFC shutdown

Nintendo announced the WFC Shutdown for May 20, 2014. Wiimm was prepared, due to his dumps, and started the server development just a few days later.

Leseratte was already involved in the mkw-ana development and helped in many ways to develop Wiimmfi. By a few days we had created a master plan:

2014-04-07: Profile id and nickname

Only the Wii and the server know the relation between the NICKNAME and the PROFILE ID (= decoded friend code). And for our own server we must have this information, because the Wii logs in with the nickname and the server has to answer with the correct profile id.

So Wiimm developed a retrieving tool that asked the GameSpy servers for the nicknames. A problem was that GameSpy only sends back the nicknames of friends. So the tool asks for friends, and then for friends of the friends, and so on.

The tool run until GameSpy server shutdown in June 2016. By the time it was shutdown, about 2.9 million nicknames for different Wii and NDS games were retrieved.

2014-04-12: NAS

NAS was the first working server. It is used for the primary logins to Wiimmfi. Nowadays it also manages bans and delayed console activation.

At this day Wiimm started with the development of the Wiimmfi Portal as an information centre.

2014-04-20: MKW patcher for the tester team

Wiimm started to develop a MKW patcher, 6 days later it was ready and served to the tester team.

The patcher created 3 images that share a new savegame. The images are for Nintendo WFC, Wiimmfi and AltWFC. So it was possible to use all 3 servers and compare dumps. The first profiles were created at Nintendo WFC and imported to Wiimmfi.

In the following 2 weeks we (the tester team) made many tests with many dumps and Wiimmfi was improved step by step. Sometimes the testers had to wait 15 to 30 minutes for a server update and the next test.

At the beginning of this phase we used the NATNEG server, created by Nagato (written in python). All other servers were written in PHP. The standalone servers (= not web servers) used a tool named NCAT (a NETCAT variant) as the main server, and a PHP script for each instance.

2014-04-28: New NATNEG server

One problem was that the MS server must send UDP/IP packets with port 27900 (owned by MASTER) as sender. It seemed to be the easier to implement it in C by creating raw UDP packets with a faked sender address.

And so Wiimm had written a new NATNEG server in C. It supports a communication channel to accept jobs for faked UDP packets. The MS server used this channel. Another advantage is that the standalone NATNEG server doesn't need a database as all protocol data is stored in an internal memory array.

From this day, the complete server software was written by Wiimm. This changed later with DLS1 and SAKE, RACE and COMPETITION support.

2014-05-03: Start of Wiimmfi (tester team only)

The tester team started to play full Mario Kart races via Wiimmfi. We tested global, regional and room races and also battles. The creation of new profiles worked too.

In the following days we tested much, and improved the server step by step.

History: Wiimmfi replaced Nintendo WFC

2014-05-10: Official start of Wiimmfi

Ten days before shutdown of Nintendo WFC, Wiimmfi was enabled for the public. At this day a generic patcher for all Wii games and a specific patcher for Mario Kart Wii were released.

2014-05-20: Shutdown of Nintendo WFC

Nintendo shut down their WFC servers. At this day, Wiimmfi already supported 17 games!

A first version of the web Mario Kart statistics was also available.

History: Wiimmfi portal

2014-09-05: Development of Wiimmfi portal

The development of the new Wiimmfi portal started. The key idea was for users to be able to login and manage their consoles and profiles. The trigger was the idea of OPENHOST.

2014-10-03: Official start of Wiimmfi portal

The new Wiimmfi portal was opened.

The Wiimmfi portal allows users to login and to manage their consoles and profiles. It also has support for a per-user rights system and was a base for the later ban system. A valid account at Wii-Homebrew.com is needed for login, as it shares a database with Wiimmfi.

2014-10-26: Naming of consoles and profiles

Consoles can be registered to a Wiimmfi account. All profiles of these consoles are listed. The user can assign names to consoles and profiles for easier identification.

OPENHOST was ready to use.

Two days later is was possible to reactivate old profiles via the portal.

2014-11-15: New Ban System

Users with a special right (moderators) can use the Wiimmfi portal to kick or ban users. All bans are logged and are publicly visible at a special ban page.

2016-08-02: New error page

The new error page is a web formular now and allows to enter a list of error codes.

For each error code, a list with class and error description is printed. For some codes, solutions to fix the error are also offered.

2016-08-20: New MKW status pages

Both Mario Kart Wii related statistic pages are updated. The region table now only shows regions with activities. Forbidden regions are also shown if used. The online table now supports 4 different views. The newest view shows a single match. It is longer available than other statstics and should be used for reports.

History: Wiimmfi servers

2014-07-05: New MASTER server

The
MASTER server has now been implemented in C as a single thread tool (multi process NCAT+PHP before). It is fast enough to handle the UDP/IP packets of more than 20000 clients.

There are many advantages with the new server:

2015-06-06: Advanced NATNEG functionality

The NATNEG server was improved to support some more NATNEG protocol variants. From now on, the NATNEG instances run at 3 different servers for improvement of connection tests of the clients.

2015-07-25: Server JOB and mkw-ana

mkw-ana is now able to connect to the new JOB server. When connected it can retrive information about the current room of the client and about other players in the same room.

mkw-ana is also able to send kick and ban requests to the JOB server. After checking rights of the user sending the requests, JOB will execute the job and thus kick or ban the player in question.

2015-12-20: Backend with TELNET support

The servers
MASTER, NATNEG and JOB now support an interactive backend. After connecting with a TELNET client, an administrator can manage the servers. The backends support a command history, a pager for long output and also a watch command for repeated status messages. The backends also support non-interactive jobs using a NCAT tool.

If you wish to test the backend, try the tool mkw-ana. It supports the same backend as the Wiimmfi servers, but with other commands.

2015-12-23: Smart restart of servers

The restart feature was finished. A restart is triggered using the
backend. When triggered, the following things happen: With this concept it is possible to restart a server with new code but without lost of any network connection or packets. The switch from the old to the new server is done in less than half a second.

2016-01-09: Watchdog for servers

The servers
MASTER, NATNEG and JOB are started by a watchdog now. If one server crashes, it is restarted immediately. The reaction time is <100 milliseconds. All servers read the database tables on watchdog restart to allow a smart continuation of the crashed server job.

Wiimm implemented the watchdogs, because the MASTER+MS crashed on an attack with invalid packets and we had to restart it manually. From the watchdog implementation, until July 2016, only one watchdog-restart for the MASTER is counted.

2016-01-30: MS integrated into MASTER

The
MS server was integrated into the MASTER server.

The combination of both servers into one process reduces the database lookups and queries again. Additionaly, the MS can now send messages as MASTER directly without using NATNEG as delivering agent.

2016-03-13: SAKE and RACE support

The SAKE and RACE servers were integrated into Wiimmfi with help by Leseratte and PokemonAcer.

SAKE is used by the games to share user-created content like ghosts and Miis in Mario Kart Wii or friend lists in Wii Speak Channel.

RACE is a Mario-Kart-specific server used to manage the in-game rankings of tracks and competitions.

2016-05-10: MKWii competitions

Exactly 2 years after Wiimmfi had publically allowed login, the competitions of Mario Kart Wii came back. Leseratte collected almost every Nintendo competition, analyzed the data and the protocol, and implemented the Wiimmfi code.

2016-09-03: New server: SV (SUPERVISOR)

Based on the code of the comming GPCM, Wiimm created a new server: SV (SUPERVISOR).

SV is the central server and has permanent connections to MASTER+MS, NATNEG, JOB, and all GPCM instances. It analyse the data of all other instances to build internal models.

Servers

NCAT : Concatenate and redirect sockets

NCAT is a feature-packed networking utility which reads and writes data across networks from the command line. It supports TCP/IP and UDP/IP as well as UNIX sockets. It is one of all NETCAT incarnations. Both tools have existed for many years.

NCAT is used as network server in combination with Wiimmfi's PHP scripts. For each connection (TCP/IP) or packet (UDP/IP), a new sub-process is created to manage the single connection or packet. So we have an own independent instance for each network client.

It was a solution to have a stable and well tested server without the need to develop and test such a management server. And when an instance crashes, it had no impact to the NCAT server or the other instances. A disadvantage is that a new sub-process is created for each network client, using more processing power.

At the beginning, NCAT was used as server for GPCM, GPSP, MASTER and MS. Nowadays it is used for GPCM, GPSP and COMPETITION.

NCAT can also used for non-interactive connections to the backends of servers. It allows a script to send commands to the servers and to collect the output.

NAS : Login server

NAS is the Nintendo Authentication Server, using HTTP/POST. Original Wii games use HTTPS, as do most DS, DSiWare and WiiWare games. Because Wiimmfi can not serve certificates signed by Nintendo, Wii games must be patched to login to Wiimmfi with HTTP and without SSL, by either a RAM patch (i.e. MrBean35000vr's patcher), or a permanent file patch (via ISO patcher).

The NAS server is written as a PHP script, run by an Apache web server. The devices send their request for authentication as POST data, and NAS replies in the usual way.

On login NAS checks parameters, maintenance mode and login restrictions like disabled games or bans. It also manages new consoles and new profiles. Login data needed by the other servers is stored into the database.

GPCM : Client server

After a succesful NAS login, the game tries to connect the GPCM server using TCP/IP, authenticating with a token from NAS. The connection will be closed with logout. When the server closes the connection, the client is kicked from Wiimmfi immediately.

At Wiimmfi, the GPCM is a combination of NCAT and PHP scripts. NCAT is the server and creates an own PHP instance for each connecting client. So each GPCM process manages exactly one client.

GPCM uses UNIX sockets for communication. The communication is needed to deliver messages between the clients (client → own GPCM → friend GPCM → friend client) and from and to other servers. The MASTER server is informed about logins and logouts for an optimized management.

Plans

Wiimm has already started to implement GPCM in C end 2015, but paused the development. The following points are already implemented but are hardly tested yet:

A main server (supervisor) manages all external connections and creates a sub-process for new clients. Each sub-process can handle any number of clients; the limit is changeable at runtime. If the limit set to NULL, the supervisor manages all clients by itself. So a single process and a multi process server is available. The single process is faster, but the multi process is more efficient and stable if one process terminates unexpectedly. So the multiprocess method was preferred at the beginning.

Each sub-process communicates with the supervisor using an unnamed socket to exchange status messages and to deliver file descriptors of clients. The supervisor redirects messages for other clients to the sub-processes. This is very fast. After sending a PING from the supervisor's backend to a sub-process the answer arrives in 50–100 microseconds. In this time the message is queued and send, the sub-process is woken up and the message received and analysed, a reply is queued and send, the supervisor is woken up and the message received and analysed and last not least a summary is printed at the backend.

The supervisor collects status data of all clients. So the supervisor knows all relevant data for statistics. It would be possible to follow all connections and disconnections of guests and hosts to create better status pages.

Data transfer with the servers MASTER and JOB is done by SEQPACKET connections. SEQPACKET is only available for UNIX sockets, and is a network stream (like TCP) with record support (like UDP). Once connected, the connection is only closed if one server is halted. At restart, the connections are established again.

GPCM allows push messages to MASTER and JOB for status changes of profiles. A test version of the JOB server already uses this to deliver some status values to mkw-ana.

The new GPCM supports a backend like the other C servers. It is available for the supervisor via such backend, and can be used to send commands to each sub-process.

OPENHOST is already implemented, but has some small issues. It needs more tests.

We tested the new GPCM at the Wiimmfi test server in March and April. Therefore, MKW Intermezzos were patched to use the test server. Overall, the new GPCM must be tested more intensively.

At the moment, the new GPCM runs as server SV (SUPERVISOR).

SV : Supervisor

SUPERVISOR (short SV) is a new analytic server. In real, it is a special working mode of the coming GPCM written in C, so it can run in the background if the new GPCM is activated.

SV is the central server and has permanent connections to MASTER+MS, NATNEG, JOB, and all GPCM instances. It analyses the data of all other instances to build internal models.

Another point is the gpcm-to-gpcm communication. Before SV, each GPCM opens a UNIX stream socket of other GPCM to deliver status messages. With the new SV, the GPCM use only one UNIX stream socket and use SV as delivery agent. Also MASTER use SV as delivery agent to e.g. kick a player.

SV has the same watchdog, restart and backend features like the other servers.

Status quo

September 2016

Plans

GPSP : Retrieving friends

GPSP is used to retrieve information about friends. The server is usually connected once after GPCM login using TCP/IP.

At Wiimmfi, GPSP is implemented as a combination of NCAT and PHP scripts. NCAT is the server and creates an own instance for each connection. Usually, a connection is closed within a second.

MASTER : Master matchmaking server

While online, each client (game) sends a UDP/IP packet with the current online status to the MASTER approximately every 30–60sec. This data is stored in a game specific database table for matchmaking, which can be retrieved by the MS server.

In the beginning, MASTER was inplemented as a PHP script. A NCAT server creates a client for each TCP/UDP packet, which results in a very low performance. The MASTER was the most database-querying server of Wiimmfi.

Very early Wiimm decided that the MASTER is the first server to be rewritten. In June 2014 he started with the development of the C server. At July 5, 2014 it replaced the old multiprocess NCAT+PHP version.

The next step was the integration of the MS server into the MASTER code. At the end of 2014 a first version was implemented and running at the test server. It took a long time until all issues are solved. So it was activated at Wiimmfi in January 2016. The combination of both servers into one process reduces the database lookups and queries even further.

In December 2015 the new backend and the restart feature were implemented. Since January 2016, watchdog will restart MASTER after a crash. Only one crash happend until July 2016.

The MASTER+MS is now fully implemented. Only special game specific features will be added if needed.

Some facts (based on an average of 150 online clients)

MS : Matchmaking retrieving server

MS is used by the clients to search for hosts for matchmaking. Answers of MS are encrypted by a game-specific secret key. If successful, MS is also used as relay for try-to-connect messages.

At Wiimmfi, MS was implemented as an NCAT and PHP script combination. NCAT is the server and creates an own instance for each connection. Usually, a connection is closed within 5 seconds.

In August 2014, Wiimm had rewritten the MS to solve different problems.

In January 2016, the PHP version was replaced by a combined MASTER+MS server written in C. The combination of both servers into one process reduces the database lookups and queries again and allows to implement special game specific features.

NATNEG : Enable client-to-client connections behind firewalls

The name NATNEG is a combination of NAT (Network Address Translation) and NEG (NEGotiation).

When playing with other players, the clients (games) must create a client-to-client connection to exchange game data. Because of firewalls and network address translations (NAT) a server must be used to open the gates. And this server is NATNEG.

At the beginning of the development, Wiimmfi used the NATNEG server of Nagato written in python, using an SQLite database.

Two weeks before official Wiimmfi start, Wiimm finished his own NATNEG server written in C. It is a standalone NATNEG server and doesn't need a database, because all protocol data is stored in an internal memory array. It also supports 2 UNIX sockets: One for the delivering of faked UDP packets and one for maintenance. The maintenance port accepts commands and sends back the output. NCAT can be used to establish the bidirectional connection.

Since June 2015, there are 3 instances of NATNEG at 3 different servers. In December 2015 the new backend and the restart feature were implemented. Since January 2016, watchdog will restart NATNEG after a crash; but a crash never happened.

Some facts (based on an average of 150 online clients)

SAKE : Server to exchange user-created content

The SAKE server is used by games to store user-created content. For Mario Kart Wii, this server stores Miis and uploaded ghost data.

Mid-2015, Leseratte used the AltWFC SAKE server to understand how basic data storage worked, and implemented bugfixes and additional features needed by Mario Kart Wii.

In March 2016, Wiimm and Leseratte reimplemented the SAKE server with all its improvements in PHP in order to use it on Wiimmfi, run by an Apache web server.

RACE : Mario-Kart-specific server for rankings

The RACE server is a Mario-Kart-specific server. The game uses this server to up- and download data to / from the in-game rankings (both for regular tracks and competitions).

The first test implementation in Python has been written by Leseratte in mid-2015, and in March 2016, at the same time as the SAKE server, it has been reimplemented in PHP by Wiimm and Leseratte, run by an Apache web server.

COMPETITION : Competition management server for Mario Kart Wii

The competition service was integrated into the RACE service on the original servers, but on Wiimmfi it has been implemented as an own service for easier development and debugging.

All the details and protocols needed for competitions were found out by Leseratte with some help by PokeAcer. Leseratte has then developed the server software needed to deliver the competitions to the Wiis, and has also improved and updated amibus first test patcher so that it now uploads old competition data to Wiimmfi and permanently patches the Wii competition download to the Wiimmfi servers.

The competition service has been developed from January 2016 to June 2016, with the first competition starting on May 10th. The server then moved from Leserattes development server to the actual Wiimmfi server in July. The PHP scripts of COMPETITION are run by an Apache web server.

JOB : Server to execute mkw-ana jobs

The JOB server is available since July 2015. It is a new server and was not available at GameSpy/Nintendo WFC, as it was written differently.

mkw-ana connects to the JOB server to get information about the current room and its players. It also sends kick and ban jobs. Special rights at the Wiimmfi server are needed for such kick and ban jobs.

In December 2015 the new backend and the restart feature were implemented. Since January 2016, watchdog will restart NATNEG after a crash; but a crash never happened.

Plans

It is planned that the JOB server sends much more information to mkw-ana after the new GPCM server is finished.

More servers ...

There are some more servers in Wiimmfi that aren't for main game play: More information will be added later.

Public Pages

User Pages