Everything You Always Wanted To Know About MMLite (but were afraid to ask)

The FAQ for MMLite

Q: Why COBs? What is the purpose of this architecture?

A: COBs (COmponent Binaries) are a way of packaging a component into a separate executable binary file. The entry point of the component (CobMain) returns an object (or NULL on failure). The COB Manager caches this object in the COB namespace and returns it to whomever binds to the name. If the COB was not already loaded the COB Manager loads it and calls the entry point.

The goal of this approach is to package a component separately from any other component and allow for easy (re)configuration. The component need not take up space in main memory until it is needed, and that space can be reclaimed when no longer in use. On the other hand, building a separate binary leads to some code duplication and extra scaffolding code. This can reduce the advantages over statically linking a component into a single executable binary together with other components.

From the docs: ) This architecture borrows a great deal from Microsoft’s COM architecture, but is not a complete COM implementation. Many of the essentials of COM are implemented in COBs and the Namespace manager: object lifetime control (reference counting) and a common, language-independent, binary interface definition. COBs can be registered and addressed within namespaces and can themselves encapsulate namespaces, house libraries, files, or device drivers.

Q: Can I write code in C++ for MMLite, and if so, what is entailed?

A: MMLite supports a subset of C++ called Embedded C++. Some C++ examples ship with the product. Embedded C++ supports many C++ features but does not support exception handling or run-time type identification. To this end, MMLite provides some (but not all) of the supporting runtime functions required by the various C++ compilers. If you wish to make use of the unsupported features you will just have to (re) implement the runtime functions yourself. This is not hard, but takes up code space.

Q: Can one write or call into VTBLs (COM objects) from C++?

A: Yes, with most C++ compilers you can call them as regular C++ objects (virtual classes). The ADS C++ compiler is the only exception. The ADS V-tables are represented in a peculiar way that makes calling between C and C++ incompatible. Even with the ADS C++ compiler you can still call methods the C way (explicit ->v and this). #define CINTERFACE to get that behavior.

Q: How do I use a scheduler other than the default (Constraint Based)?

A: This is set at build time. See the makefiles in conf/package (where components are packaged together into images). You will select the scheduler that you desire by replacing the references to cb_sched.lib.

Q: How do I use a heap other than the default (First Fit)?

A: Same as above, replace the references to ff_heap.lib.

You can also use any type of heap dynamically, by calling the *HeapNew() function. For example, if you want a Bitmap heap, you can call BitmapHeapNew(). However, to do so, you must first allocate memory from the default heap. There are also examples that package the heap managers separately as COBs, look in the src\heaps directory.

Q: How do I write a device driver for MMLite?

A: Same as in any other OS, copy and modify an existing one!

The base system only provides support for installing, removing, and invoking an Interrupt Service Routine (ISR). The rest is up to you. We do provide examples and configuration code like in other OSes. Our examples are real-time safe and generally rather minimalistic.

Q: How do I put a new build on my EB63 board?

A: Run EB_FLASH.EXE to copy the binaries and file system over. Look at the atmel.htm file for more detailed instructions.

Q: How do I build the MMLite binaries?

A: The top level mkall.bat file builds everything that we know is in a consistent state. The results of the build are in the build subdirectories, one per architecture/toolset combination. The build directory also contains C header files that are generated from the XML specifications. You might want to look in there for some definitions that perhaps you expected to find in the top level include directory.

Q: Can I use the .NET compiler ?

You can build through the Visual Studio 7.0 tool set. To do this, install VS7.0, run VCVARS32.BAT, then run NMAKE from the topmost source directory. Visual Studio 6.0 works the same way but you also need the Platform SDK, available from MSDN -- the environment variable MsSdk says where it is.

Q: What kind of debug support do I have for MMLite in general?

A: You can debug in one of several ways:

  • You can run under the emulator, NTU.EXE, and debug applications through the Microsoft Developer Studio IDE (assuming you are writing code in a supported language). This is by far the easiest way to develop code on MMLite. In general, one should write and debug code on NTU, making occasional sanity tests on the hardware platform.
  • You can use the GNU debugger (GDB) to debug through the serial port, or over a network link. The GNU debugger has bugs, but can be used to get a stack trace and peek at variables.
  • You can use the ARM emulation / debug environment, ARM-Ulator. If you have a lot of money you can use the Multi-ICE.
  • You can put print statements in your code.
  • Q: How do I make a program run automagically at startup?

    A: There are several ways:

  • You can edit the base source code, setting the first thread (function FirstThread) to be your code, and link the code into the base image _fredEb63.bin. This is useful if you are rather constrained in memory footprint and are trying to eliminate all unnecessary code, such as the loader.
  • You can compile a program into a binary named INIT.EXE, and put it in the fs file system. At boot time, MMLite will load and execute that file.
  • You can use the TZK method : in this case, the INIT.EXE program is a simple shell, TZK.EXE (renamed INIT.EXE), which searches for a file called INIT.TZK in the fs namespace. INIT.TZK is a text file containing the name of the binary you wish to run.
  • Q: What image formats are supported ?

    We provide loader code for ARM's AIF format, the ELF format, and the Microsoft PE format. We have code for other proprietary formats that we cannot distribute. You can write your own loader using the examples in src\loaders.

    Q: What is the TZK shell syntax for MMLite?

    A: Simple : each line is read in order. If it starts with a bang (!) it is a comment, and is ignored. Otherwise it is read and executed as if it were typed in at the command line. You must use the full file name (for example, foo.exe not just foo to run program foo), and it is case-sensitive. NTU runs on Windows and is therefore case-insensitive, but this is a property of that particular filesystem and not of the command line interpreter. Use the help command for a list of commands that TZK understands.

    Q: How can I write and run a shell script for MMLite?

    A: Write it in TZK format (above) with your favorite text editor. Run it by typing ‘source foo.tzk’ or ‘. foo.tzk’, where foo.tzk contains your script.

    Q: How do I build a new file system image for MMLite? I just want to have a certain set of files and directories present at boot time – I do not need to construct a file system driver like DOS-FAT, NTFS, or HPFS.

    A: This is done at build time, under Windows. There is a program called MKMEMFS.EXE (see how it is used in the makefiles in conf\package). It builds a filesystem image that is understood by the ROMFs filesystem code. Usually you will put that image in FLASH memory, and tell the ROMFs constructor about it, either the builtin version which is conditionally enabled in src\base\md\arm\eb63\_first.c or using romfstest.exe.

    Q: I want to port MMLite onto another platform. Is this possible? How much work is entailed?

    A: Yes, we tried to make that as easy as we possibly could. First you need to compile the machine-independent MMLite source code using a compiler that supports your chosen processor. The code is ANSI-C and has been handled by a large number of compilers so this should not be a problem.

    Then you need to chose the machine dependent code for your processor, and possibly your compiler. Under the /src directory, you will find several subdirectories, each owning an /md subdirectory in turn. In src/crt/md you will find all the machine dependent code for the C Runtime libraries, 64-bit math and any special compiler support.

    Next you will create a subdirectory under src/base/md for your platform. That directory is organized by processor types to encourage code sharing and reuse. Look at the arm directory for some guidance. In general, you should identify and properly isolate the code as handling all hardware dependent functions of the OS: Interrupt handling and atomic increment and decrement functions are examples. We do not have examples of support for floating point math, as it is rare in the embedded world to find such coprocessors.

    Finally, you will adapt some existing device driver for your platform, or write new ones. Look at the serial line driver for one possible way to organize your driver for portability.

    If you are familiar with MMLite and your system, and you are an expert system programmer it should not take you more than say about a week to get this all up and running. Have fun and let us know how you liked it!

    Q: Can I use UNICODE ?

    A: Yes. We build and test a version of NTU with UNICODE enabled. However, we never used it anywhere else so you might find bugs lurking. For instance, some runtime functions are known to be missing.

    Q: What inter-process communications facilities are available on MMLite?

    Within the typical single physical address space of an embedded processor the answer is simple: use the Namespace to register and lookup objects across process boundaries. We use this just about everywhere. If you mean across virtual memory boundaries the answer is none at this time. If you mean across machine boundaries we have support for SOAP, even if it is a bit shakey at the moment.

    Q: Can I use the Berkeley / Winsock sockets interface on MMLite?

    A: Yes. MMLite supports the IEndpoint interface which is essentially a COM wrapper around the Berkeley sockets layer. MMLite also include WSOCK32.LIB providing the regular Berkeley sockets APIs (as of Winsock 1.1).

    Q: What synchronization primitives are available on MMLite?

    A: The blocking primitives supported are Mutexes and Condition Variables (Windows NT programmers know condition variables as events). Non-blocking primitives are fully supported, such as CompareAndSwap, Atomic increment, decrement, addition, and subtraction. You can build anything else with these.

    Semaphores are not supported directly, and neither are critical sections, although critical sections can be emulated with a mutex. There is a thread-safe list with insertion and removal functions (AtomicQueues).

    Q: Does MMLite support Microsoft’s COM/DCOM server?

    A: No. But the COB (Component Object Binary) format supports a subset of COM functionality, specifically, interface inheritance and versioning through the VTBL interface. You can write a COB object knowing that future enhancements to the object (through the addition of interfaces) will not break existing clients. Moreover, you can be certain that the interface is language independent, so clients and server (COB) objects need not be written in the same language.

    Q: How do I detect a thread exit on MMLite?

    A: If you are searching for an analogy to NT’s WaitForSingleObject(hThread, …), you search in vain. To synchronize on a thread, one must use mutexes and condition variables. Threads exist but not as synchronization objects (you cannot wait for a thread to finish like you can in NT).

    There are many examples in the test directory of programs written with multiple threads. Barrier synchronization is one popular way. In general, writing code to handle your threads instead of handling your data is a poor, non-scalable, error-prone way to do things.

    Q: How do I tell when my process can safely terminate without marooning thread resources?

    A: You can detect process completion with the Module->WaitFor() method. A thread takes a reference on the process object which creates it, and releases it when the thread terminates. This means the process object cannot be destroyed by the system until every thread created in that process has ended. Threads also have refcounts, which are decremented when the thread exits, and which protect thread resources from harvest until thread termination. Currently processes and modules are the same thing. Use QueryInterface to get one from the other.

    Q: Can I obtain a mutex recursively (on MMLite) ?

    A: No. Mutexes are standard (POSIX) and non-recursive objects. If you try to do so you will deadlock immediately, your thread will lock and never return from the Mutex_Lock call. Recursive locks have been avoided by in the standard because they are well known to be error-prone.

    It is possible to implement any other synchronization primitive using Mutexes and Condition variables so you can add your favorite, see sample.

    Q: Does MMLite support virtual memory?

    A: It is prototype stage, and is not shipped or supported.

    Q: Does MMLite support memory protection? Do MMLite processes run in a protected “sandbox”, or can one MMLite process forge a pointer into another process’ address space?

    A: It does not support memory protection. On the other hand, the chip sets we support in general do not support address protection at the hardware level anyway.

    Q: What does the MM in MMLite stand for?

    A: Multimedia, but it is historical and misleading …

    Q: Can I boot my IBM PC clone to MMLite?

    A: Not shipped or supported.

    Q: I want to build a parallel architecture based on MMLite. Is this remotely possible (far less supported)?

    A: Remotely, yes. The MMLite base was architected to support parallelism, but as of press time (March 2002), MMLite has yet to boot on a parallel system. To implement MMLite in parallel, one must first implement an interrupt arbitration scheme – that is, one must design and develop a means by which hardware interrupts are delegated to specific CPUs. As this is hardware dependent, it is not a part of the MMLite base. In addition to interrupt arbitration, the MMLite schedulers do not at this time support multiple processors in an efficient manner (although they should work at a very basic level).

    Q: What are the legal terms for the use of MMLite?

    A: In general, MMLite can be used for noncommercial purposes. Any commercial use must be licensed by Microsoft. The specifics are at http://research.microsoft.com/invisible/EULA.htm

    Q: MMLite supports multiple heaps, and multiple types of heaps. What are the supported heap types, and what are the [dis]advantages of each?

    A: FirstFit, Bitmap, Temporary, Debug and Null.

  • The FirstFit strategy has been proved to be near-optimal in the literature. FirstFit is a general-purpose heap which provides good performance under all conditions, and avoids bad worst-case performance. It limits fragmentation by buddy-merging freed blocks. The overheads are per-used-block and therefore depend on usage. This general purpose heap is comparable in performance to those used by all other general purpose operating systems, the footprint is probably smaller but it really depends on what you compare it with.
  • The Bitmap heap has a fixed overhead (the allocation bitmap) that depends only on the total amount of memory managed, not by how it is used. Its performance is superior to the FirstFit heap in the presence of caches, because the metadata is not spread around in memory (linked lists) but represented compactly in the allocation bitmap. There are however degenerate use cases that would lead to long searches of the bitmap, such as the case where every-other-sequential-block is in use. Therefore this is not selected as the default system heap. This heap has also an interesting special feature, unlike other heaps it is able to free memory based only on a pointer to the middle of a block. For example, network drivers typically allocate a block for a network frame, which comprises one or more headers plus a payload. As the frame is passed around the protocol stack, the pointer is incremented past the headers. At the end of the stack, it must be freed – which means the bottom of the stack has to know where the frame block began.
  • The Null heap can allocate memory, but cannot free it, and is the heap to use if you have an application that needs essentially static amounts of memory. The Null heap takes a very small footprint, which makes it attractive for very small and simple systems.
  • The Temporary heap can allocate memory, but can only free it by destroying the entire heap. For example, a SOAP parser may need to allocate memory to parse a new block of XML. When it has finished parsing, it knows it is done with that block, and can zorch it all in one fell stroke.
  • The Debug heap is a wrapper for the system heap which checks for memory corruptions and out-of-bounds accesses.
  • Q: Can one process contain two heaps of different types simultaneously?

    A: Yes. However, for each process there is only one CurrentHeap (the default heap). More heaps can be created given memory. E.g. allocate some memory out of the default heap and turn it into another heap using the heap classe's constructor (you can call it as a COB, see src/heaps/new_heap.c).

    Q: Why does the GNU debugger (GDB) fail to recognize my executable?

    A: Probably you are using a version configured to look for another kind of executable. You need to configure your debugger to look for the type you are using. For the EB63, that is most likely ARM-ELF. It does not support ARM AIF that we know of.

    Q: How do I run NTU?

    A: This is just a regular Win32 console application, run it from any command line window. It takes as arguments the name of the first program to run (that is, instead of init.exe) and the arguments for it. The directory you are in when you run NTU becomes the fs/ directory so it is best to cd to the correct place first: e.g. cd build\i386\debug\bin.

    Q: Why does my compiler choke when I make an assignment to a variable of type TIME?

    A: TIME is defined as a 64 bit integer. If your compiler does not support 64 bit math, or if you are compiling with 64 bit math disabled (default for GNU-C compilation in MMLite), 64 bit integers are defined (in MMLite and other systems) as a structure of smaller integers, probably two 32 bit integers. So you may see errors when making assignments to TIME typed values because you are really operating on structures not integers. The workaround is to use the 64-bit arithmetic macros to write code that is portable with and without compiler support. Another workaround, of course, is to enable 64-bit arithmetic if it is available.

    Q: Why do I get a funky string on the screen as soon as I hit ‘reset’?

    A: You are hitting a debug break in your boot sequence. The string you see is the GNU debug sync string – it is intended to permit GDB to connect for debugging. This means your file system contains a version of init.tzk (the startup file, analogous to AUTOEXEC.BAT or .cshrc) which is starting an executable that bugchecks, either by calling DebugBreak() or by doing something bad like dereferencing a NULL pointer.