Difference between revisions of "MemHack"

From Screeps Wiki
Jump to navigation Jump to search
(Added the words 'Memory Hack' so that the wiki search will actually produce this page.)
 
(5 intermediate revisions by 2 users not shown)
Line 1: Line 1:
memHack refers to the strategy of storing a user's <code>Memory</code> object in <code>global</code> to skip the <code>JSON.parse()</code> normally required at first-access of <code>Memory</code> in a tick. Normally, the way <code>Memory</code> works is when first accessed in a tick all of <code>Memory</code> is <code>JSON.parsed()</code> for the tick it is being used, then after any changes have been made end-of-tick it is <code>JSON.stringify()</code> ed for storage for the next tick it is accessed.  The user running their code pays for this parsing/stringifying with [[CPU]] the larger the <code>Memory</code> object the more costly it is to process.  So, memHack works by creating an object to store <code>Memory</code> in global when global is 'first created' (and needs to be rebuilt if global dies due to code push, global reset, ect), then at the start of a tick (start of a user's main loop) deletes the 'normal' <code>Memory</code> object and replaces it with the global version which due to global persisting between ticks does not need to be parsed.  It finally sets the <code>RawMemory._parsed</code> to the current global memHack obj, this insures changes are updated to the normal <code>Memory</code> object, so that in the event of a global reset when reconstructing the <code>Memory</code> object will still have the changes and that the Memory watcher / console changing still work.  It is important to note, that <code>RawMemory._parsed</code> is not officially documented in the API however it has been around and accessible for several years.  It is possible for this method to become defunct in the future.
+
MemHack (''Memory Hack'') refers to the strategy of storing a user's <code>Memory</code> object in <code>global</code> to skip the <code>JSON.parse()</code> normally required at first-access of <code>Memory</code> in a tick, thus saving [[CPU]] every tick. Note that the hack is exactly that: a hack. It relies on the underlying implementation of the API, not the published API itself, giving it the potential to become obsolete or broken in the future. 
 +
 
 +
== Memory Implementation ==
 +
The way the <code>Memory</code> object works is when first accessed in a game tick it is run though a <code>JSON.parse()</code> which converts it to an object from its stored-as-a-string state, then after any changes have been made end-of-tick it is run though a <code>JSON.stringify()</code> for storage.  The user running the code pays for this parsing/stringifying with [[CPU]]: the larger the <code>Memory</code> object, the more costly it is to process
 +
 
 +
When Memory is parsed naturally, the parsed object is stored in <code>RawMemory._parsed</code>At the end of the tick (after your code ends), the engine serializes the value in <code>RawMemory._parsed</code> into the database, for use in the next tick.  The time taken for serialization is included in your CPU usage.  You can read more about the <code>Memory</code> object in [https://docs.screeps.com/global-objects.html#Memory-object Screep's official docs]. 
 +
 
 +
== Operation ==
 +
MemHack works by creating an object in <code>global</code> to store the <code>Memory</code> object when initializing the global (and needs to be rebuilt if global dies due to code push, global reset, ect). At the start of subsequent ticks (start of a user's main loop before any other <code>Memory</code> related calls) deletes the 'normal' <code>Memory</code> object and replaces it with the global version which due to global persisting between ticks does not need to be parsed.  Then it sets <code>RawMemory._parsed</code> to the current global MemHack obj as well, this insures changes are updated to the normal <code>Memory</code> object, so that in the event of a global reset the <code>Memory</code> object will still have the changes and, that the Memory watcher / console changing still work.  It is important to note, that <code>RawMemory._parsed</code> is not officially documented in the API however, it has been around and accessible for several years.  It is possible for this strategy to become defunct in the future, should this be changed.
 +
 
 +
== Risky Business ==
 +
If you are not risk adverse or want additional savings, you can delete <code>RawMemory._parsed</code> for a tick, preventing serialization for that tick.  You can add a control structure using <code>Game.time % x === 0</code> to have it only serialize every 5, 10, etc ticks.  The lower the gap between saves, the easier it would be to recover from the older copy of Memory serialized in the backend.  This could lose data such as recent [[Market]] transactions, intelligence gathered from [[StructureObserver|Observers]], and other things.
  
 
== Code Examples ==
 
== Code Examples ==
 
There are a few places you can view examples of this code:
 
There are a few places you can view examples of this code:
  
[https://github.com/screepers/screeps-snippets/blob/master/src/misc/JavaScript/Memory%20Cache.js postCrafter - Screep's Snippets]
+
[https://github.com/screepers/screeps-snippets/blob/master/src/misc/JavaScript/Memory%20Cache.js postCrafter - Screepers' Snippets]
  
 
[https://github.com/ags131/ZeSwarm/blob/v1.1/src/MemHack.js ags131 - ZeSwarm]
 
[https://github.com/ags131/ZeSwarm/blob/v1.1/src/MemHack.js ags131 - ZeSwarm]
  
 
[https://github.com/bencbartlett/Overmind/blob/master/src/memory/Memory.ts#L83 bencbartlett - OverMind]
 
[https://github.com/bencbartlett/Overmind/blob/master/src/memory/Memory.ts#L83 bencbartlett - OverMind]

Latest revision as of 02:09, 30 July 2021

MemHack (Memory Hack) refers to the strategy of storing a user's Memory object in global to skip the JSON.parse() normally required at first-access of Memory in a tick, thus saving CPU every tick. Note that the hack is exactly that: a hack. It relies on the underlying implementation of the API, not the published API itself, giving it the potential to become obsolete or broken in the future.

Memory Implementation[edit | edit source]

The way the Memory object works is when first accessed in a game tick it is run though a JSON.parse() which converts it to an object from its stored-as-a-string state, then after any changes have been made end-of-tick it is run though a JSON.stringify() for storage. The user running the code pays for this parsing/stringifying with CPU: the larger the Memory object, the more costly it is to process.

When Memory is parsed naturally, the parsed object is stored in RawMemory._parsed. At the end of the tick (after your code ends), the engine serializes the value in RawMemory._parsed into the database, for use in the next tick. The time taken for serialization is included in your CPU usage. You can read more about the Memory object in Screep's official docs.

Operation[edit | edit source]

MemHack works by creating an object in global to store the Memory object when initializing the global (and needs to be rebuilt if global dies due to code push, global reset, ect). At the start of subsequent ticks (start of a user's main loop before any other Memory related calls) deletes the 'normal' Memory object and replaces it with the global version which due to global persisting between ticks does not need to be parsed. Then it sets RawMemory._parsed to the current global MemHack obj as well, this insures changes are updated to the normal Memory object, so that in the event of a global reset the Memory object will still have the changes and, that the Memory watcher / console changing still work. It is important to note, that RawMemory._parsed is not officially documented in the API however, it has been around and accessible for several years. It is possible for this strategy to become defunct in the future, should this be changed.

Risky Business[edit | edit source]

If you are not risk adverse or want additional savings, you can delete RawMemory._parsed for a tick, preventing serialization for that tick. You can add a control structure using Game.time % x === 0 to have it only serialize every 5, 10, etc ticks. The lower the gap between saves, the easier it would be to recover from the older copy of Memory serialized in the backend. This could lose data such as recent Market transactions, intelligence gathered from Observers, and other things.

Code Examples[edit | edit source]

There are a few places you can view examples of this code:

postCrafter - Screepers' Snippets

ags131 - ZeSwarm

bencbartlett - OverMind