QisMLib Web Page Header

Memory Maps

In order to display or otherwise work with a GDSII file, QISLIB must first scan it to determine the contents -- cells, references, entities and so on. It then makes a second pass and loads into memory a quad tree which divides the layout into regions and contains pointers to entity data in the actual file. The entities may remain on disk or be loaded into RAM (depending on whether there is enough RAM to hold the entities.) This scanning of the file and building the quad tree and associated pointers can take a significant amount of time for large files.

If one opens the same file over and over again (in different sessions) the scan/load time becomes a significant overhead. The overhead can be eliminated by producing a memory map -- essentially a copy of what QisMLib places into memory upon scanning/loading a file.

File loading process requires significant computation before data is available for exploding.

Fig. 1 - when opening a GDSII file, significant computation must be performed. However it is possible to save the results of those computations to a "memory map" for use later.


Once the memory map has been produced (and saved to disk) the next time the user wishes to display or otherwise process the GDSII layout, the memory map is loaded directly -- no need for any scanning of the input file or any building of the quad trees. The time required is really only limited by the disk IO read time.

using a memory map for loading greatly decreases the load time since there are fewer computations to do.

Fig. 2 - using a memory map for loading greatly decreases the load time since there are fewer computations to do.


Memory Maps with Entity Data

In the illustrations above, we produced only the scan map and load map. All of the entity data (i.e. the boundaries, paths, text ) contained in the cell definitions is not put into the memory map. Why? Well if the GDSII entity data is so large it won't fit into the computer's memory then it is left on disk and when loading/processing the file, the entity data is read directly from the source GDSII file.

While this is slower than if the entity data is stored in RAM, it does allow the library to handle very large files.

If you know that your computer has sufficient RAM to store the entity map in memory (and it is compressed as it is read into memory) then you can choose to include this data as part of your dbload file.

First, when producing the memory map use the option to include the entity data.

producing memory map with entity data.

Fig. 3 - Producing a memory map with entity data.


Reading a memory map with entity data.

Fig. 4 - Reading a memory map with entity data.


OASIS Limitation

Memory maps can only be produced for GDSII stream files -- not for OASIS files. The reasons for this are complex and related to how we organize OASIS data vs GDSII data. So the following instructions apply solely to GDSII input.


Generating the Memory Map

There are two methods of creating memory maps:
  1. use a command line utility included in the QisMLib package.
  2. have the client use the appropriate API calls

(1) Command Line

gds2dbload +gds:<GDSII-INPUT> +outdir:<OUTPUT-DIR> [OPTIONS]

where



gds:<GDSII-INPUT>              Specify the path of a valid GDSII file to be used
                                     as a source for the .dbload [REQD]

outdir:<OUTPUT-DIR>            Specify the path of an existing directory with 
                               write permissions where the .scan.l64 and 
                               .dbload.l64 files will be created [REQD]

                               The output file name format is 
                               <GDSII-FILE-NAME>.scan.l64 
                               and <GDSII-FILE-NAME>.dbload.l64. 
                               eg a GDSII file demo.gds will be converted as 
                               demo.gds.scan.l64 and demo.gds.dbload.l64

                               These .l64 file names should not be modified and
                               should be present in the same directory at all 
                               times.

                               The version of the .dbload file is same as the 
                               major.minor version of gds2dbload. 
                               E.g gds2dbload v1.6.xx will create 
                               .dbload.l64 v1.06


OPTIONS


-layers:<LAYER-LIST>          Specify a list of layer(s) or layer:datatype(s) 
                              to extract. If not used (by default), all the 
                              layers present in the GDSII file will be 
                              extracted as-is.

                              If the :datatype is omitted for a layer, all 
                              datatypes for that layer are extracted.
                              eg : -layers:1,2:10,3,4:20 will extract all 
                              datatypes for layers 1 and 3 and only datatypes 
                              10 and 20 for layers 2 and 4 respectively. 
                              Data on all other layers and datatypes will 
                              be dropped

-notext                       If specified, no text elements are extracted. 
                              [DEFAULT] text is extracted as-is.

-igntxtscal                   If specified, text scale information will not
                              be used to compute cell extents. Any cell 
                              containing only TEXT will have zero extents 
                              and therefore it will be considered empty.

-loademptyrefs                If specified, empty cells (cells with zero 
                              extents) are loaded.



-log:<LOG-FILE>               Create an execution log at the specified path 
                              (including file name.) 
                              The directory component of the path must exist 
                              with suitable write permissions. 
                              [DEFAULT] no log is created.
-OR-

-log+:<LOG-FILE>              If -log+ is used, an exisiting file at that path 
                              will be appended. With -log, an exisiting file will be 
                              overwritten.

  
-silent                       If used, no messages are written to stdout/stderr.
                              If not used [DEFAULT], the execution log is 
                              written to stdout and error/warning messages 
                              are written to stderr.


(2) Using the QisMFile Class: QisMFileMemoryMaps



virtual bool SaveAs_memory_maps(const char* outputDir, QisMNotify* progressCb)=0;

where

outputDir                     Path of the directory where the memory maps 
                              will be created (If NULL or empty, they will 
                              be created in the same location as the input 
                              file)


progressCb                    Address of a handler for receiving progress 
                              updates.

RETURNS
true                          Success

false                         Failure
                              If false, Call Get_last_error_msg or 
                              Get_last_error_code to get information
                              about this error.

NOTES
                              If the source file was loaded to memory i.e. 
                              (QisMFileLoadCtrl::Get_file_data_on_disk() == false), 
                              .scan and .dbload memory maps will be created. 
                              Otherwise, .scan and .load memory maps will
                              be created.


                              Pre-load settings such as layer-mapping/filtering 
                              (QisMFileLoadCtrl::Set_layer_map)or dropping texts 
                              (QisMFileLoadCtrl::Ignore_texts() == true) 
                              will be reflected in the newly created memory 
                              maps.

                              This method is intended primarily for created 
                              .scan/.dbload memory maps since they can be used 
                              independently of the original source file ,