GEGELATI

Loading...
Searching...
No Matches
Public Member Functions | Static Protected Member Functions | List of all members
TPG::ExecutionStats Class Reference

Utility class for extracting execution statistics from a TPGExecutionEngineInstrumented and an instrumented TPGGraph. More...

`#include <executionStats.h>`

## Public Member Functions
| ExecutionStats ()=default
| Default constructor.
virtual | ~ExecutionStats ()=default
| Default destructor.
void | analyzeInstrumentedGraph (const TPGGraph *graph)
| Analyze the average statistics of an instrumented TPGGraph execution.
void | analyzeInferenceTrace (const std::vector< const TPGVertex * > &trace)
| Analyze the execution statistics of one inference trace.
void | analyzeExecution (const TPG::TPGExecutionEngineInstrumented &tee, const TPGGraph *graph)
| Analyze the execution statistics of multiple inferences done with a TPGExecutionEngineInstrumented.
double | getAvgEvaluatedTeams () const
| Get the average number of evaluated teams per inference.
double | getAvgEvaluatedPrograms () const
| Get the average number of programs evaluated per inference.
double | getAvgExecutedLines () const
| Get the average number of executed lines per inference.
const std::map< size_t, double > & | getAvgNbExecutionPerInstruction () const
const std::vector< TraceStats > & | getInferenceTracesStats () const
| Get stored trace statistics.
const std::map< size_t, size_t > & | getDistribEvaluatedTeams () const
| Get the distribution of the number of evaluated teams.
const std::map< size_t, size_t > & | getDistribEvaluatedPrograms () const
| Get the distribution of the number of evaluated programs.
const std::map< size_t, size_t > & | getDistribExecutedLines () const
| Get the distribution of the number of executed lines.
const std::map< size_t, std::map< size_t, size_t > > & | getDistribNbExecutionPerInstruction () const
| Get distributions of the number of executions for each instruction.
const std::map< const TPG::TPGVertex *, size_t > & | getDistribUsedVertices () const
| Get the distribution if the number of visit for each vertex.
void | clearInferenceTracesStats ()
| Clear stored trace statistics and distributions.
void | writeStatsToJson (const char *filePath, bool noIndent=false) const
| Export the execution statistics of the last analyzeExecution() call to a file using Json format.

## Static Protected Member Functions
static void | analyzeProgram (std::map< uint64_t, uint64_t > &instructionCounts, const Program::Program &program)
| Analyze a program to get how many times each instruction is used.

## Detailed Description

Utility class for extracting execution statistics from a TPGExecutionEngineInstrumented and an instrumented TPGGraph.

The main method of this class is analyzeExecution(), which will :

- retrieve from a TPGGraph the instrumented values and compute average execution statistics.
- compute execution statistics for every inference done with a TPGExecutionEngineInstrumented.
- create distributions from statistics of analyzed traces.

Before analyzing or even starting any inference, you must :

- use a TPGGraph associated to a TPGInstrumentedFactory.
- use a TPGExecutionEngineInstrumented that will execute the TPGGraph.
- clear any previous instrumented data : --> for the TPGGraph, use TPGInstrumentedFactory::resetTPGGraphCounters(). --> for the TPGExecutionEngineInstrumented, use its method TPGExecutionEngineInstrumented::clearTraceHistory(). Otherwise, the results won't have any meaning. If you have never executed the TPGGraph or the TPGExecutionEngineInstrumented, resetting them isn't required.

You can then execute the TPG for as many inferences as you like.

Then, use analyzeExecution() with the TPGGraph and TPGExecutionEngineInstrumented, and access the statistics using the provided getters and setters.

Warning : the class deduces the number of inferences based on the sum of evaluation each root vertices had. If you executed your TPGGraph starting from multiple roots, then remember that the average statistics are based on ALL inferences, regardless of the root vertices used.

The Json exporter is designed to be used after a call to analyzeExecution(). Just call writeStatsToJson() to export statistics in a file with json format.

## Member Function Documentation

## ◆ analyzeExecution()

void TPG::ExecutionStats::analyzeExecution | ( | const TPG::TPGExecutionEngineInstrumented & | tee,
|  | const TPGGraph * | graph )

Analyze the execution statistics of multiple inferences done with a TPGExecutionEngineInstrumented.

Previous results will be erased.

Parameters

  [in] | tee | the TPGExecutionEngineInstrumented.
  [in] | graph | the TPGGraph executed with tee.

Exceptions

  std::bad_cast | if the graph contains a non instrumented vertex or edge.

## ◆ analyzeInferenceTrace()

void TPG::ExecutionStats::analyzeInferenceTrace | ( | const std::vector< const TPGVertex * > & | trace | ) |

Analyze the execution statistics of one inference trace.

The vector trace contains all visited vertices for one inference in order : trace[0] is the root, and trace[trace.size()-1] the action.

Results are stored in a new TraceStats struct which is pushed back in attribute inferenceTracesStats. Previous results will be erased.

Parameters

  [in] | trace | a vector<const TPGVertex*> of the analyzed inference trace.

## ◆ analyzeInstrumentedGraph()

void TPG::ExecutionStats::analyzeInstrumentedGraph | ( | const TPGGraph * | graph | ) |

Analyze the average statistics of an instrumented TPGGraph execution.

Results are stored in the average results class attributes.

Parameters

  [in] | graph | the analyzed TPGGraph*.

Exceptions

  std::bad_cast | if graph contains at least one non instrumented vertex or edge.

## ◆ analyzeProgram()

void TPG::ExecutionStats::analyzeProgram | ( | std::map< uint64_t, uint64_t > & | instructionCounts,
|  | const Program::Program & | program )
| staticprotected

Analyze a program to get how many times each instruction is used.

Parameters

  [out] | instructionCounts | the std::map<uint64_t, uint64_t>& that will be incremented for each instruction use.
  [in] | program | the analyzed program.

Copyright or © or Copr. IETR/INSA - Rennes (2022) :

Elinor Montmasson elino.nosp@m.r.mo.nosp@m.ntmas.nosp@m.son@.nosp@m.gmail.nosp@m..com (2022)

GEGELATI is an open-source reinforcement learning framework for training artificial intelligence based on Tangled Program Graphs (TPGs).

This software is governed by the CeCILL-C license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL-C license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info".

As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability.

In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security.

The fact that you are presently reading this means that you have had knowledge of the CeCILL-C license and that you accept its terms.

## ◆ getAvgNbExecutionPerInstruction()

const std::map< size_t, double > & TPG::ExecutionStats::getAvgNbExecutionPerInstruction | ( |  | ) | const

Get a reference to the map that associate each instruction to its average number of execution per inference.

## ◆ writeStatsToJson()

void TPG::ExecutionStats::writeStatsToJson | ( | const char * | filePath,
|  | bool | noIndent = false ) const

Export the execution statistics of the last analyzeExecution() call to a file using Json format.

This method will use the statistics currently stored in the object because it is intended to be used after a call to analyzeExecution(). Using it after separate calls to analyzeInstrumentedGraph() or analyzeInferenceTrace() might lead to uncorrelated data.

Data is organized as follows :

     {
         "ExecutionStats" :
         {
             "avgEvaluatedTeams" : value,
             "avgEvaluatedPrograms" : value,
             "avgExecutedLines" : value,
             "avgNbExecutionPerInstruction" :
             {
                 "InstructionIndex" : nbExecution,
                 ...
             },
             "distributionEvaluatedPrograms" :
             {
                 "N" : count of inferences which evaluated N

programs, ... }, "distributionEvaluatedTeams" : { "N" : count of inferences which evaluated N teams, ... }, "distributionExecutedLines" : { "N" : count of inferences which executed N lines, ... }, "distributionNbExecutionPerInstruction" : { "InstructionIndex" : { "N" : count of inferences which executed the instruction N times, ... }, ... }, "distributionUsedVertices" : { "VertexIndex" : count of inferences which visited the vertex, ... } },

"TracesStats" : { "TraceNumber" : { "nbEvaluatedPrograms" : value, "nbEvaluatedTeams" : value, "nbExecutedLines" : value, "nbExecutionPerInstruction" : { "InstructionIndex" : nbExecution, ... }, "trace" : [Array of vertex indexes in the TPGGraph] }, ... }

}

Parameters

  [in] | filePath | the path to the output file.
  [in] | noIndent | true if the json format must not be indented. Files can become large quickly with a lot of traces, so if the file will just be analyzed by another program, set this to true to save some space on your disk. Set noIndent to false if you want to keep the file readable.

The documentation for this class was generated from the following files:

- D:/a/gegelati/gegelati/gegelatilib/include/tpg/instrumented/executionStats.h
- D:/a/gegelati/gegelati/gegelatilib/src/tpg/instrumented/executionStats.cpp