UnmarshalStream Class Reference

Class which turns marshal bytecode into Python object. More...

#include "EVEUnmarshal.h"

Collaboration diagram for UnmarshalStream:

Public Member Functions
	UnmarshalStream ()
	~UnmarshalStream ()
PyRep *	Load (const Buffer &data)
	Loads Python object from given bytecode. More...

Protected Member Functions
template<typename T >
const T &	Peek () const
template<typename T >
Buffer::const_iterator< T >	Peek (size_t count) const
template<typename T >
const T &	Read ()
template<typename T >
Buffer::const_iterator< T >	Read (size_t count)
uint32	ReadSizeEx ()
PyRep *	LoadStream (size_t streamLength)
PyRep *	LoadRep ()
void	CreateObjectStore (size_t streamLength, uint32 saveCount)
	Initializes object store. More...
void	DestroyObjectStore ()
	Destroys object store. More...
uint32	GetStorageIndex ()
	Obtains storage index for StoreObject. More...
PyRep *	GetStoredObject (uint32 index)
	Obtains previously stored object. More...
void	StoreObject (uint32 index, PyRep *object)
	Stores object. More...

Private Member Functions
PyRep *	LoadNone ()
PyRep *	LoadBoolTrue ()
PyRep *	LoadBoolFalse ()
PyRep *	LoadIntegerLongLong ()
PyRep *	LoadIntegerLong ()
PyRep *	LoadIntegerSignedShort ()
PyRep *	LoadIntegerByte ()
PyRep *	LoadIntegerVar ()
PyRep *	LoadIntegerMinusOne ()
PyRep *	LoadIntegerZero ()
PyRep *	LoadIntegerOne ()
PyRep *	LoadReal ()
PyRep *	LoadRealZero ()
PyRep *	LoadStringEmpty ()
PyRep *	LoadStringChar ()
PyRep *	LoadStringShort ()
PyRep *	LoadStringLong ()
PyRep *	LoadStringTable ()
PyRep *	LoadWStringEmpty ()
PyRep *	LoadWStringUCS2Char ()
PyRep *	LoadWStringUCS2 ()
PyRep *	LoadWStringUTF8 ()
PyRep *	LoadToken ()
PyRep *	LoadBuffer ()
PyRep *	LoadTupleEmpty ()
PyRep *	LoadTuple ()
PyRep *	LoadTupleOne ()
PyRep *	LoadTupleTwo ()
PyRep *	LoadListEmpty ()
PyRep *	LoadList ()
PyRep *	LoadListOne ()
PyRep *	LoadDict ()
PyRep *	LoadObject ()
PyRep *	LoadObjectEx1 ()
PyRep *	LoadObjectEx2 ()
PyRep *	LoadSubStream ()
PyRep *	LoadSubStruct ()
PyRep *	LoadChecksumedStream ()
PyRep *	LoadPackedRow ()
PyRep *	LoadError ()
PyRep *	LoadSavedStreamElement ()
PyObjectEx *	LoadObjectEx (bool is_type_2)
bool	LoadRLE (Buffer &out)

Private Attributes
Buffer::const_iterator< uint8 >	mInItr
Buffer::const_iterator< uint32 >	mStoreIndexItr
PyList *	mStoredObjects

Static Private Attributes
static PyRep *(UnmarshalStream::*const	s_mLoadMap [])()

Detailed Description

Class which turns marshal bytecode into Python object.

Author

Zhur, Bloody.Rabbit

Definition at line 54 of file EVEUnmarshal.h.

Constructor & Destructor Documentation

UnmarshalStream::UnmarshalStream ( )

inline

Definition at line 57 of file EVEUnmarshal.h.

    : mStoredObjects( nullptr )
    {
    }

UnmarshalStream::~UnmarshalStream

(

)

Definition at line 58 of file EVEUnmarshal.cpp.

{
    //PySafeDecRef(mStoredObjects);
}

Member Function Documentation

void UnmarshalStream::CreateObjectStore ( size_t  streamLength, uint32  saveCount  )

protected

Initializes object store.

Parameters

[in]	streamLength	Length of stream.
[in]	saveCount	Number of saved objects within the stream.

Definition at line 181 of file EVEUnmarshal.cpp.

References DestroyObjectStore(), mInItr, mStoredObjects, and mStoreIndexItr.

Referenced by LoadStream().

{
    DestroyObjectStore();

    if( 0 < saveCount )
    {
        mStoreIndexItr = ( ( mInItr + streamLength ).As<uint32>() - saveCount );
        mStoredObjects = new PyList( saveCount );
    }
}

Here is the call graph for this function:

Here is the caller graph for this function:

void UnmarshalStream::DestroyObjectStore ( )

protected

Destroys object store.

Definition at line 192 of file EVEUnmarshal.cpp.

References mStoredObjects, mStoreIndexItr, and PySafeDecRef.

Referenced by CreateObjectStore(), and LoadStream().

{
    mStoreIndexItr = Buffer::const_iterator<uint32>();
    PySafeDecRef( mStoredObjects );
}

Here is the caller graph for this function:

uint32 UnmarshalStream::GetStorageIndex ( )

inlineprotected

Obtains storage index for StoreObject.

Returns

Storage index.

Definition at line 126 of file EVEUnmarshal.h.

References mStoreIndexItr.

Referenced by LoadRep().

{ return *mStoreIndexItr++; }

Here is the caller graph for this function:

PyRep * UnmarshalStream::GetStoredObject ( uint32  index )

protected

Obtains previously stored object.

Parameters

[in]

index

Index of stored object.

Returns

The stored object.

Definition at line 198 of file EVEUnmarshal.cpp.

References PyList::GetItem(), and mStoredObjects.

Referenced by LoadSavedStreamElement().

{
    if( 0 < index )
        return mStoredObjects->GetItem( --index );
    return nullptr;
}

Here is the call graph for this function:

Here is the caller graph for this function:

PyRep * UnmarshalStream::Load

(

const Buffer &

data

)

Loads Python object from given bytecode.

Parameters

[in]

data

Buffer containing marshal bytecode.

Returns

Loaded Python object.

Definition at line 134 of file EVEUnmarshal.cpp.

References Buffer::begin(), LoadStream(), mInItr, and Buffer::size().

Referenced by Unmarshal().

{
    mInItr = data.begin<uint8>();
    PyRep* res = LoadStream( data.size() );
    mInItr = Buffer::const_iterator<uint8>();

    return res;
}

Here is the call graph for this function:

Here is the caller graph for this function:

PyRep* UnmarshalStream::LoadBoolFalse ( )

inlineprivate

Loads false boolean from stream.

Definition at line 150 of file EVEUnmarshal.h.

References PyStatic.

{ return PyStatic.NewFalse(); }

PyRep* UnmarshalStream::LoadBoolTrue ( )

inlineprivate

Loads true boolean from stream.

Definition at line 148 of file EVEUnmarshal.h.

References PyStatic.

{ return PyStatic.NewTrue(); }

PyRep * UnmarshalStream::LoadBuffer ( )

private

Loads buffer from stream.

Definition at line 328 of file EVEUnmarshal.cpp.

References ReadSizeEx().

{
    const uint32 len = ReadSizeEx();
    const Buffer::const_iterator<uint8> data = Read<uint8>( len );

    return new PyBuffer( data, data + len );
}

Here is the call graph for this function:

PyRep * UnmarshalStream::LoadChecksumedStream ( )

private

Loads checksumed stream from stream.

Definition at line 495 of file EVEUnmarshal.cpp.

References LoadRep().

{
    const uint32 sum = Read<uint32>();

    PyRep* ss = LoadRep();
    if( NULL == ss )
        return nullptr;

    return new PyChecksumedStream( ss, sum );
}

Here is the call graph for this function:

PyRep * UnmarshalStream::LoadDict ( )

private

Loads dict from stream.

Definition at line 418 of file EVEUnmarshal.cpp.

References key(), LoadRep(), PyDecRef, ReadSizeEx(), and PyDict::SetItem().

{
    const uint32 count = ReadSizeEx();
    PyDict* dict = new PyDict;

    for ( uint32 i(0); i < count; i++ )
    {
        PyRep* value = LoadRep();
        if( NULL == value )
            return nullptr;

        PyRep* key = LoadRep();
        if( NULL == key )
        {
            PyDecRef( value );
            return nullptr;
        }

        dict->SetItem( key, value );
    }

    return dict;
}

Here is the call graph for this function:

PyRep * UnmarshalStream::LoadError ( )

private

Prints error and returns NULL.

Definition at line 692 of file EVEUnmarshal.cpp.

References sLog.

{
    sLog.Error( "Unmarshal", "Invalid opcode encountered." );

    return nullptr;
}

PyRep* UnmarshalStream::LoadIntegerByte ( )

inlineprivate

Loads byte integer from stream.

Definition at line 159 of file EVEUnmarshal.h.

{ return new PyInt( Read<int8>() ); }

PyRep* UnmarshalStream::LoadIntegerLong ( )

inlineprivate

Loads long integer from stream.

Definition at line 155 of file EVEUnmarshal.h.

{ return new PyInt( Read<int32>() ); }

PyRep* UnmarshalStream::LoadIntegerLongLong ( )

inlineprivate

Loads long long integer from stream.

Definition at line 153 of file EVEUnmarshal.h.

{ return new PyLong( Read<int64>() ); }

PyRep* UnmarshalStream::LoadIntegerMinusOne ( )

inlineprivate

Loads minus one integer from stream.

Definition at line 163 of file EVEUnmarshal.h.

{ return new PyInt( -1 ); }

PyRep* UnmarshalStream::LoadIntegerOne ( )

inlineprivate

Loads one integer from stream.

Definition at line 167 of file EVEUnmarshal.h.

{ return new PyInt( 1 ); }

PyRep* UnmarshalStream::LoadIntegerSignedShort ( )

inlineprivate

Loads signed short from stream.

Definition at line 157 of file EVEUnmarshal.h.

{ return new PyInt( Read<int16>() ); }

PyRep * UnmarshalStream::LoadIntegerVar ( )

private

Loads variable length integer from stream.

Definition at line 214 of file EVEUnmarshal.cpp.

References ReadSizeEx().

{
    /* this is one of the stranger fields I have found, it seems to be a variable
     * length integer field (somewhat of a 'bigint' style data type), but it gets
     * used at times for integers which would fit into the other primitive data
     * types.... I would really like to see the criteria they use to determine
     * what gets marshaled as what...
     */

    const uint32 len = ReadSizeEx();
    const Buffer::const_iterator<uint8> data = Read<uint8>( len );

    if( sizeof( int32 ) >= len )
    {
        int32 intval(0);
        memcpy( &intval, &*data, len );

        return new PyInt( intval );
    }
    else if( sizeof( int64 ) >= len )
    {
        int64 intval(0);
        memcpy( &intval, &*data, len );

        return new PyLong( intval );
    }
    else
    {
        //int64 is not big enough
        //just pass it up to the application layer as a buffer...
        return new PyBuffer( data, data + len );
    }
}

Here is the call graph for this function:

PyRep* UnmarshalStream::LoadIntegerZero ( )

inlineprivate

Loads zero integer from stream.

Definition at line 165 of file EVEUnmarshal.h.

{ return new PyInt( 0 ); }

PyRep * UnmarshalStream::LoadList ( )

private

Loads list from stream.

Definition at line 386 of file EVEUnmarshal.cpp.

References LoadRep(), PyDecRef, ReadSizeEx(), and PyList::SetItem().

{
    const uint32 count = ReadSizeEx();
    PyList* list = new PyList( count );

    for ( uint32 i(0); i < count; i++ )
    {
        PyRep* rep = LoadRep();
        if (rep == nullptr)
        {
            PyDecRef( list );
            return nullptr;
        }

        list->SetItem( i, rep );
    }

    return list;
}

Here is the call graph for this function:

PyRep* UnmarshalStream::LoadListEmpty ( )

inlineprivate

Loads empty list from stream.

Definition at line 210 of file EVEUnmarshal.h.

{ return new PyList( 0 ); }

PyRep * UnmarshalStream::LoadListOne ( )

private

Loads one-element list from stream.

Definition at line 406 of file EVEUnmarshal.cpp.

References PyList::AddItem(), and LoadRep().

{
    PyRep* i = LoadRep();
    if( NULL == i )
        return nullptr;

    PyList* list = new PyList();
    list->AddItem( i );

    return list;
}

Here is the call graph for this function:

PyRep* UnmarshalStream::LoadNone ( )

inlineprivate

Loads none from stream.

Definition at line 145 of file EVEUnmarshal.h.

References PyStatic.

{ return PyStatic.NewNone(); }

PyRep * UnmarshalStream::LoadObject ( )

private

Loads object from stream.

Definition at line 442 of file EVEUnmarshal.cpp.

References PyRep::AsString(), PyRep::IsString(), LoadRep(), PyDecRef, sLog, and PyRep::TypeString().

{
    PyRep* type = LoadRep();
    if( NULL == type )
        return nullptr;

    if( !type->IsString() )
    {
        sLog.Error( "Unmarshal", "Object: Expected 'String' as type, got '%s'.", type->TypeString() );

        PyDecRef( type );
        return nullptr;
    }

    PyRep* arguments = LoadRep();
    if( NULL == arguments )
    {
        PyDecRef( type );
        return nullptr;
    }

    return new PyObject( type->AsString(), arguments );
}

Here is the call graph for this function:

PyObjectEx * UnmarshalStream::LoadObjectEx ( bool  is_type_2 )

private

Helper; loads extended object from stream.

Definition at line 713 of file EVEUnmarshal.cpp.

References PyList::AddItem(), PyObjectEx::dict(), key(), PyObjectEx::list(), LoadRep(), Op_PackedTerminator, PyDecRef, and PyDict::SetItem().

Referenced by LoadObjectEx1(), and LoadObjectEx2().

{
    PyRep* header = LoadRep();
    if( NULL == header )
        return nullptr;

    PyObjectEx* obj = new PyObjectEx( is_type_2, header );

    while( Op_PackedTerminator != Peek<uint8>() )
    {
        PyRep* el = LoadRep();
        if( NULL == el )
        {
            PyDecRef( obj );
            return nullptr;
        }

        obj->list().AddItem( el );
    }
    //skip Op_PackedTerminator
    Read<uint8>();

    while( Op_PackedTerminator != Peek<uint8>() )
    {
        PyRep* key = LoadRep();
        if( NULL == key )
        {
            PyDecRef( obj );
            return nullptr;
        }

        PyRep* value = LoadRep();
        if( NULL == value )
        {
            PyDecRef( key );
            PyDecRef( obj );
            return nullptr;
        }

        obj->dict().SetItem( key, value );
    }
    //skip Op_PackedTerminator
    Read<uint8>();

    return obj;
}

Here is the call graph for this function:

Here is the caller graph for this function:

PyRep * UnmarshalStream::LoadObjectEx1 ( )

private

Loads extended object of type 1 from stream.

Definition at line 466 of file EVEUnmarshal.cpp.

References LoadObjectEx().

{
    return LoadObjectEx( false );
}

Here is the call graph for this function:

PyRep * UnmarshalStream::LoadObjectEx2 ( )

private

Loads extended object of type 2 from stream.

Definition at line 471 of file EVEUnmarshal.cpp.

References LoadObjectEx().

{
    return LoadObjectEx( true );
}

Here is the call graph for this function:

PyRep * UnmarshalStream::LoadPackedRow ( )

private

Loads packed row from stream.

Definition at line 506 of file EVEUnmarshal.cpp.

References Buffer::const_iterator< T >::As(), Buffer::begin(), DBRowDescriptor::ColumnCount(), DBTYPE_BOOL, DBTYPE_BYTES, DBTYPE_CY, DBTYPE_EMPTY, DBTYPE_ERROR, DBTYPE_FILETIME, DBTYPE_GetSizeBits(), DBTYPE_I1, DBTYPE_I2, DBTYPE_I4, DBTYPE_I8, DBTYPE_R4, DBTYPE_R8, DBTYPE_STR, DBTYPE_UI1, DBTYPE_UI2, DBTYPE_UI4, DBTYPE_UI8, DBTYPE_WSTR, DBRowDescriptor::GetColumnType(), PyPackedRow::header(), LoadRep(), LoadRLE(), PyDecRef, and PyPackedRow::SetField().

{
    // PyPackedRows are just a packed form of blue.DBRow
    // these take a DBRowDescriptor and the column data in different formats
    PyRep* header_element = LoadRep();
    if( NULL == header_element )
        return nullptr;

    // create the base packed row to be filled with data
    PyPackedRow* row = new PyPackedRow( (DBRowDescriptor*)header_element );

    // create the sizemap and sort it by bitsize, the value of the map indicates the index of the column
    // this can be used to identify things easily
    std::multimap< uint8, uint32, std::greater< uint8 > > sizeMap;
    std::map<uint8,uint8> booleanColumns;

    uint32 columnCount = row->header()->ColumnCount();
    size_t byteDataBitLength = 0;
    size_t booleansBitLength = 0;
    size_t nullsBitLength = 0;

    for (uint32 i(0); i < columnCount; i++ )
    {
        DBTYPE columnType = row->header()->GetColumnType (i);
        uint8_t size = DBTYPE_GetSizeBits (columnType);

        // count booleans
        if (columnType == DBTYPE_BOOL)
        {
            booleanColumns.insert (std::make_pair (i, booleansBitLength));
            booleansBitLength++;
        }

        // count all columns as possible nulls
        nullsBitLength ++;

        // increase the bytedata length only if a column is longer than 7 bits
        // this is used as an indicator of what is written in the first, second or third part
        if (size >= 8)
            byteDataBitLength += size;

        // add the column to the list
        sizeMap.insert (std::make_pair (size, i));
    }

    size_t expectedByteSize = (byteDataBitLength >> 3) + ((booleansBitLength + nullsBitLength) >> 3) + 1;

    // reserve enough space for the buffer
    Buffer unpacked (expectedByteSize, 0);

    if( !LoadRLE(unpacked) )
    {
        PyDecRef( header_element );
        return nullptr;
    }

    Buffer::const_iterator<uint8> unpackedItr = unpacked.begin<uint8>();
    Buffer::const_iterator<uint8> bitIterator = unpacked.begin<uint8>();

    std::multimap< uint8, uint32, std::greater< uint8 > >::iterator cur, end;
    cur = sizeMap.begin();
    end = sizeMap.end();
    for (; cur != end; ++cur)
    {
        const uint32 index = cur->second;
        const DBTYPE columnType = row->header ()->GetColumnType (index);

        unsigned long nullBit = byteDataBitLength + booleansBitLength + cur->second;
        unsigned long nullByte = nullBit >> 3;
        // setup the iterator to the proper byte
        // first check for nulls
        bitIterator = unpacked.begin<uint8>() + nullByte;

        if ((*bitIterator & (1 << (nullBit & 0x7))) == (1 << (nullBit & 0x7)))
        {
            // PyNone value found! override it and increase the original iterator the required steps
            unpackedItr += DBTYPE_GetSizeBits (columnType) >> 3;
            row->SetField (index, new PyNone ());

            // continue should only be performed if the columns are not normal marshal objects
            if (columnType != DBTYPE_BYTES && columnType != DBTYPE_STR && columnType != DBTYPE_WSTR)
                continue;
        }

        switch (columnType)
        {
            case DBTYPE_I8:
            case DBTYPE_CY:
            case DBTYPE_UI8:
            case DBTYPE_FILETIME:
            {
                Buffer::const_iterator<int64> v = unpackedItr.As<int64>();
                row->SetField( index, new PyLong( *v++ ) );
                unpackedItr = v.As<uint8>();
            } break;

            case DBTYPE_I4:
            {
                Buffer::const_iterator<int32> v = unpackedItr.As<int32>();
                row->SetField( index, new PyInt( *v++ ) );
                unpackedItr = v.As<uint8>();
            } break;
            case DBTYPE_UI4:
            {
                Buffer::const_iterator<uint32> v = unpackedItr.As<uint32>();
                row->SetField( index, new PyInt( *v++ ) );
                unpackedItr = v.As<uint8>();
            } break;

            case DBTYPE_I2:
            {
                Buffer::const_iterator<int16> v = unpackedItr.As<int16>();
                row->SetField( index, new PyInt( *v++ ) );
                unpackedItr = v.As<uint8>();
            } break;
            case DBTYPE_UI2:
            {
                Buffer::const_iterator<uint16> v = unpackedItr.As<uint16>();
                row->SetField( index, new PyInt( *v++ ) );
                unpackedItr = v.As<uint8>();
            } break;

            case DBTYPE_I1:
            {
                Buffer::const_iterator<int8> v = unpackedItr.As<int8>();
                row->SetField( index, new PyInt( *v++ ) );
                unpackedItr = v.As<uint8>();
            } break;

            case DBTYPE_UI1:
            {
                Buffer::const_iterator<uint8> v = unpackedItr.As<uint8>();
                row->SetField( index, new PyInt( *v++ ) );
                unpackedItr = v.As<uint8>();
            } break;

            case DBTYPE_R8:
            {
                Buffer::const_iterator<double> v = unpackedItr.As<double>();
                row->SetField( index, new PyFloat( *v++ ) );
                unpackedItr = v.As<uint8>();
            } break;

            case DBTYPE_R4:
            {
                Buffer::const_iterator<float> v = unpackedItr.As<float>();
                row->SetField( index, new PyFloat( *v++ ) );
                unpackedItr = v.As<uint8>();
            } break;

            case DBTYPE_BOOL:
            {
                // get the bit this boolean should be read from
                unsigned long boolBit = byteDataBitLength + booleanColumns.find (index)->second;
                unsigned long boolByte = boolBit >> 3;
                // setup the iterator to the proper byte
                bitIterator = unpacked.begin<uint8>() + boolByte;

                row->SetField (index, new PyBool ((*bitIterator & (1 << (boolBit & 0x7))) == (1 << (boolBit & 0x7))));
            } break;

            // these objects are read directly from the end of the PyPackedRow
            // so they can be kept
            case DBTYPE_BYTES:
            case DBTYPE_STR:
            case DBTYPE_WSTR:
            {
                PyRep* el = LoadRep();
                if( NULL == el )
                {
                    PyDecRef( row );
                    return nullptr;
                }

                row->SetField( index, el );
            } break;

            case DBTYPE_EMPTY:
            case DBTYPE_ERROR:
                return nullptr;
        }
    }

    return row;
}

Here is the call graph for this function:

PyRep* UnmarshalStream::LoadReal ( )

inlineprivate

Loads real from stream.

Definition at line 170 of file EVEUnmarshal.h.

{ return new PyFloat( Read<double>() ); }

PyRep* UnmarshalStream::LoadRealZero ( )

inlineprivate

Loads zero real from stream.

Definition at line 172 of file EVEUnmarshal.h.

{ return new PyFloat( 0.0 ); }

PyRep * UnmarshalStream::LoadRep ( )

protected

Loads rep from stream.

Definition at line 160 of file EVEUnmarshal.cpp.

References GetStorageIndex(), PyRepOpcodeMask, PyRepSaveMask, PyRepUnknownMask, s_mLoadMap, sLog, and StoreObject().

Referenced by LoadChecksumedStream(), LoadDict(), LoadList(), LoadListOne(), LoadObject(), LoadObjectEx(), LoadPackedRow(), LoadStream(), LoadSubStruct(), LoadTuple(), LoadTupleOne(), and LoadTupleTwo().

{
    const uint8 header = Read<uint8>();

    const bool flagUnknown = ( header & PyRepUnknownMask ) != 0;
    const bool flagSave = ( header & PyRepSaveMask ) != 0;
    const uint8 opcode = ( header & PyRepOpcodeMask );

    if( flagUnknown )
        sLog.Warning( "Unmarshal", "Encountered flagUnknown in header 0x%X.", header );

    const uint32 storageIndex = ( flagSave ? GetStorageIndex() : 0 );

    PyRep* rep = ( this->*s_mLoadMap[ opcode ] )();

    if( 0 != storageIndex )
        StoreObject( storageIndex, rep );

    return rep;
}

Here is the call graph for this function:

Here is the caller graph for this function:

bool UnmarshalStream::LoadRLE ( Buffer &  out )

private

Helper; loads zero-compressed buffer from stream.

Definition at line 760 of file EVEUnmarshal.cpp.

References ReadSizeEx(), and Buffer::size().

Referenced by LoadPackedRow().

{
    const uint32 in_size = ReadSizeEx();

    Buffer::const_iterator<uint8> cur, end;
    cur = Read<uint8>(in_size );
    end = cur + in_size;
    Buffer::const_iterator<uint8> in_ix = cur;
    int out_ix = 0;
    int count;
    int run = 0;
    int nibble = 0;

    while(in_ix < end)
    {
        nibble = !nibble;
        if(nibble)
        {
            run = (unsigned char)*in_ix++;
            count = (run & 0x0f) - 8;
        }
        else
            count = (run >> 4) - 8;

        if(count >= 0)
        {
            if (out_ix + count + 1 > out.size())
                return false;

            while(count-- >= 0)
                out[out_ix++] = 0;
        }
        else
        {
            if (out_ix - count > out.size())
                return false;

            while(count++ && in_ix < end)
                out[out_ix++] = *in_ix++;
        }
    }

    // no need to set the rest of the buffer to zero as the output is already
    // set to 0
    // while(out_ix < out.size())
    //    out[out_ix++] = 0;

    return true;
}

Here is the call graph for this function:

Here is the caller graph for this function:

PyRep * UnmarshalStream::LoadSavedStreamElement ( )

private

Loads saved stream element from stream.

Definition at line 699 of file EVEUnmarshal.cpp.

References PyRep::Clone(), GetStoredObject(), ReadSizeEx(), and sLog.

{
    const uint32 index = ReadSizeEx();

    PyRep* obj = GetStoredObject( index );
    if( NULL == obj )
    {
        sLog.Error( "Unmarshal", "SavedStreamElement: Got invalid stored object." );
        return nullptr;
    }

    return obj->Clone();
}

Here is the call graph for this function:

PyRep * UnmarshalStream::LoadStream ( size_t  streamLength )

protected

Initializes loading and loads rep from stream.

Definition at line 143 of file EVEUnmarshal.cpp.

References CreateObjectStore(), DestroyObjectStore(), LoadRep(), MarshalHeaderByte, and sLog.

Referenced by Load().

{
    const uint8 header = Read<uint8>();
    if (MarshalHeaderByte != header) {
        sLog.Error( "Unmarshal", "Invalid stream received (header byte 0x%X).", header );
        return nullptr;
    }

    const uint32 saveCount = Read<uint32>();
    CreateObjectStore( streamLength - sizeof( uint8 ) - sizeof( uint32 ), saveCount );

    PyRep* rep = LoadRep();

    DestroyObjectStore();
    return rep;
}

Here is the call graph for this function:

Here is the caller graph for this function:

PyRep * UnmarshalStream::LoadStringChar ( )

private

Loads single character string from stream.

Definition at line 248 of file EVEUnmarshal.cpp.

{
    const Buffer::const_iterator<char> str = Read<char>( 1 );

    return new PyString( str, str + 1 );
}

PyRep* UnmarshalStream::LoadStringEmpty ( )

inlineprivate

Loads empty string from stream.

Definition at line 175 of file EVEUnmarshal.h.

{ return new PyString( "" ); }

PyRep * UnmarshalStream::LoadStringLong ( )

private

Loads long (no limit) string from stream.

Definition at line 263 of file EVEUnmarshal.cpp.

References ReadSizeEx().

{
    const uint32 len = ReadSizeEx();
    const Buffer::const_iterator<char> str = Read<char>( len );

    return new PyString( str, str + len );
}

Here is the call graph for this function:

PyRep * UnmarshalStream::LoadStringShort ( )

private

Loads short (up to 255 chars) string from stream.

Definition at line 255 of file EVEUnmarshal.cpp.

{
    const uint8 len = Read<uint8>();
    const Buffer::const_iterator<char> str = Read<char>( len );

    return new PyString( str, str + len );
}

PyRep * UnmarshalStream::LoadStringTable ( )

private

Loads table string from stream.

Definition at line 271 of file EVEUnmarshal.cpp.

References sLog, sMarshalStringTable, and snprintf.

{
    const uint8 index = Read<uint8>();

    const char* str = sMarshalStringTable.LookupString( index );
    if( NULL == str )
    {
        assert( false );
        sLog.Error( "Unmarshal", "String Table Item %u is out of range!", index );

        char ebuf[64];
        snprintf( ebuf, 64, "Invalid String Table Item %u", index );
        return new PyString( ebuf );
    }
    else
        return new PyString( str );
}

PyRep * UnmarshalStream::LoadSubStream ( )

private

Loads sub stream from stream.

Definition at line 476 of file EVEUnmarshal.cpp.

References ReadSizeEx().

{
    const uint32 len = ReadSizeEx();
    const Buffer::const_iterator<uint8> data = Read<uint8>( len );

    return new PySubStream( new PyBuffer( data, data + len ) );
}

Here is the call graph for this function:

PyRep * UnmarshalStream::LoadSubStruct ( )

private

Loads sub struct from stream.

Definition at line 484 of file EVEUnmarshal.cpp.

References LoadRep().

{
    // This is actually a remote object specification

    PyRep* ss = LoadRep();
    if( NULL == ss )
        return nullptr;

    return new PySubStruct( ss );
}

Here is the call graph for this function:

PyRep * UnmarshalStream::LoadToken ( )

private

Loads token from stream.

Definition at line 320 of file EVEUnmarshal.cpp.

{
    const uint8 len = Read<uint8>();
    const Buffer::const_iterator<char> str = Read<char>( len );

    return new PyToken( str, str + len );
}

PyRep * UnmarshalStream::LoadTuple ( )

private

Loads tuple from stream.

Definition at line 336 of file EVEUnmarshal.cpp.

References LoadRep(), PyDecRef, ReadSizeEx(), and PyTuple::SetItem().

{
    const uint32 count = ReadSizeEx();
    PyTuple* tuple = new PyTuple( count );

    for ( uint32 i(0); i < count; ++i ) {
        PyRep* rep = LoadRep();
        if (rep == nullptr) {
            PyDecRef( tuple );
            return nullptr;
        }

        tuple->SetItem( i, rep );
    }

    return tuple;
}

Here is the call graph for this function:

PyRep* UnmarshalStream::LoadTupleEmpty ( )

inlineprivate

Loads empty tuple from stream.

Definition at line 201 of file EVEUnmarshal.h.

{ return new PyTuple( 0 ); }

PyRep * UnmarshalStream::LoadTupleOne ( )

private

Loads one-element tuple from stream.

Definition at line 354 of file EVEUnmarshal.cpp.

References LoadRep(), and PyTuple::SetItem().

{
    PyRep* i = LoadRep();
    if( NULL == i )
        return nullptr;

    PyTuple* tuple = new PyTuple( 1 );
    tuple->SetItem( 0, i );

    return tuple;
}

Here is the call graph for this function:

PyRep * UnmarshalStream::LoadTupleTwo ( )

private

Loads two-element tuple from stream.

Definition at line 366 of file EVEUnmarshal.cpp.

References LoadRep(), PyDecRef, and PyTuple::SetItem().

{
    PyRep* i = LoadRep();
    if( NULL == i )
        return nullptr;

    PyRep* j = LoadRep();
    if( NULL == j )
    {
        PyDecRef( i );
        return nullptr;
    }

    PyTuple *tuple = new PyTuple( 2 );
    tuple->SetItem( 0, i );
    tuple->SetItem( 1, j );

    return tuple;
}

Here is the call graph for this function:

PyRep* UnmarshalStream::LoadWStringEmpty ( )

inlineprivate

Loads empty wide string from stream.

Definition at line 186 of file EVEUnmarshal.h.

{ return new PyWString( "", 0 ); }

PyRep * UnmarshalStream::LoadWStringUCS2 ( )

private

Loads UCS-2 string from stream.

Definition at line 300 of file EVEUnmarshal.cpp.

References ReadSizeEx(), and utf16to8().

{
    const uint32 len = ReadSizeEx();
    const Buffer::const_iterator<uint16> wstr = Read<uint16>( len );

    // convert to UTF-8
    std::string str;
    utf8::utf16to8( wstr, wstr + len, std::back_inserter( str ) );

    return new PyWString( str );
}

Here is the call graph for this function:

PyRep * UnmarshalStream::LoadWStringUCS2Char ( )

private

Loads single UCS-2 character string from stream.

Definition at line 289 of file EVEUnmarshal.cpp.

References utf16to8().

{
    const Buffer::const_iterator<uint16> wstr = Read<uint16>( 1 );

    // convert to UTF-8
    std::string str;
    utf8::utf16to8( wstr, wstr + 1, std::back_inserter( str ) );

    return new PyWString( str );
}

Here is the call graph for this function:

PyRep * UnmarshalStream::LoadWStringUTF8 ( )

private

Loads UTF-8 string from stream.

Definition at line 312 of file EVEUnmarshal.cpp.

References ReadSizeEx().

{
    const uint32 len = ReadSizeEx();
    const Buffer::const_iterator<char> wstr = Read<char>( len );

    return new PyWString( wstr, wstr + len );
}

Here is the call graph for this function:

template<typename T >

const T& UnmarshalStream::Peek ( ) const

inlineprotected

Peeks element from stream.

Definition at line 76 of file EVEUnmarshal.h.

{ return *Peek<T>( 1 ); }

template<typename T >

Buffer::const_iterator<T> UnmarshalStream::Peek ( size_t  count ) const

inlineprotected

Peeks elements from stream.

Definition at line 79 of file EVEUnmarshal.h.

References Buffer::const_iterator< T >::As(), and mInItr.

{ return mInItr.As<T>(); }

Here is the call graph for this function:

template<typename T >

const T& UnmarshalStream::Read ( )

inlineprotected

Reads element from stream.

Definition at line 83 of file EVEUnmarshal.h.

{ return *Read<T>( 1 ); }

template<typename T >

Buffer::const_iterator<T> UnmarshalStream::Read ( size_t  count )

inlineprotected

Reads elements from stream.

Definition at line 86 of file EVEUnmarshal.h.

References mInItr.

    {
        Buffer::const_iterator<T> res = Peek<T>( count );
        mInItr = ( res + count ).template As<uint8>();
        return res;
    }

uint32 UnmarshalStream::ReadSizeEx ( )

inlineprotected

Reads extended size from stream.

Definition at line 94 of file EVEUnmarshal.h.

Referenced by LoadBuffer(), LoadDict(), LoadIntegerVar(), LoadList(), LoadRLE(), LoadSavedStreamElement(), LoadStringLong(), LoadSubStream(), LoadTuple(), LoadWStringUCS2(), and LoadWStringUTF8().

    {
        uint32 size = Read<uint8>();
        if( 0xFF == size )
            size = Read<uint32>();

        return size;
    }

Here is the caller graph for this function:

void UnmarshalStream::StoreObject ( uint32  index, PyRep *  object  )

protected

Stores object.

Parameters

[in]	index	Index of object.
[in]	object	The object to be stored.

Definition at line 205 of file EVEUnmarshal.cpp.

References mStoredObjects, PyIncRef, and PyList::SetItem().

Referenced by LoadRep().

{
    if( 0 < index )
    {
        PyIncRef( object );
        mStoredObjects->SetItem( --index, object );
    }
}

Here is the call graph for this function:

Here is the caller graph for this function:

Member Data Documentation

Buffer::const_iterator<uint8> UnmarshalStream::mInItr

private

Buffer iterator we are processing.

Definition at line 247 of file EVEUnmarshal.h.

Referenced by CreateObjectStore(), Load(), Peek(), and Read().

PyList* UnmarshalStream::mStoredObjects

private

Referenced objects within the buffer.

Definition at line 252 of file EVEUnmarshal.h.

Referenced by CreateObjectStore(), DestroyObjectStore(), GetStoredObject(), and StoreObject().

Buffer::const_iterator<uint32> UnmarshalStream::mStoreIndexItr

private

Next store index for referencing in the buffer.

Definition at line 250 of file EVEUnmarshal.h.

Referenced by CreateObjectStore(), DestroyObjectStore(), and GetStorageIndex().

PyRep*( UnmarshalStream::* const UnmarshalStream::s_mLoadMap[])()

staticprivate

Load function map.

Definition at line 255 of file EVEUnmarshal.h.

Referenced by LoadRep().

---

The documentation for this class was generated from the following files:

• /backups/local/src/eve/EvEmu_Crucible/src/eve-common/marshal/EVEUnmarshal.h
• /backups/local/src/eve/EvEmu_Crucible/src/eve-common/marshal/EVEUnmarshal.cpp