Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/improve analytics logs #510

Merged
merged 27 commits into from
May 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
0fd6ae6
Add ILoggableExpression
dom-apuliasoft May 2, 2024
4182437
Add custom loggable name to bifs
dom-apuliasoft May 2, 2024
09a32d5
Add support for time tracking based on program
dom-apuliasoft May 2, 2024
54405f4
Merge branch 'develop' into feature/improve-analytics-logs
dom-apuliasoft May 2, 2024
5a8fd44
Add in extension function String.outputOf a callback that allows to c…
lanarimarco May 3, 2024
cb28dde
Fix some formatting issues
dom-apuliasoft May 3, 2024
16437af
Merge branch 'develop' into feature/improve-analytics-logs
dom-apuliasoft May 13, 2024
4b60d91
Refactor program time collection
dom-apuliasoft May 13, 2024
571b39c
Fix ktlintCheck
dom-apuliasoft May 13, 2024
cb48818
Fix outdated test
dom-apuliasoft May 13, 2024
f9ef48f
Avoid duplicate ANALYTICS output
dom-apuliasoft May 13, 2024
e89e416
Fix missing position for certain expressions
dom-apuliasoft May 14, 2024
61a4551
Add separator between timestamp and channel on log render
dom-apuliasoft May 14, 2024
df1b0d0
Add support for nested statements logging
dom-apuliasoft May 14, 2024
eadb0d6
Fix pattern in LoggingTest
dom-apuliasoft May 14, 2024
bf67090
Add support for nested expressions in analytics
dom-apuliasoft May 15, 2024
7712030
Implement ANALYTICS nested expressions logging
dom-apuliasoft May 15, 2024
ca5ed1a
Merge branch 'develop' into feature/improve-analytics-logs
dom-apuliasoft May 15, 2024
5dd3ff0
Move analytics records to performance logs creation
dom-apuliasoft May 22, 2024
aca6fc1
Add some perf logging test examples
lanarimarco May 23, 2024
e93ef3e
Cleanup
dom-apuliasoft May 24, 2024
7103381
Merge branch 'develop' into feature/improve-analytics-logs
dom-apuliasoft May 24, 2024
8ad5a64
Move SymbolTable logging logic
dom-apuliasoft May 24, 2024
9d604c0
Update logging documentation
dom-apuliasoft May 24, 2024
6c84525
Disable afterSystemInterface
dom-apuliasoft May 24, 2024
52c5ce1
Fix issue with afterSystemInterface
dom-apuliasoft May 28, 2024
55a7813
Merge branch 'develop' into feature/improve-analytics-logs
dom-apuliasoft May 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 31 additions & 22 deletions docs/logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -201,10 +201,10 @@ The log records are generated at the end of statements block, measuring the
time in microseconds.

```
15:51:55.838 PERF MULANGTC10 7 PLIST elapsed 8.875us
15:51:55.840 PERF MULANGTC10 13 EVAL elapsed 566us
15:51:55.840 PERF MULANGTC10 14 EVAL elapsed 35.084us
+---------------- header -----------------+--------+----- perf ----+
13:21:41.347 PERF MULANGTC10 11 DataRefExpr 60
13:21:41.347 PERF MULANGTC10 11 %CHAR 504
13:21:41.347 PERF MULANGTC10 11 PlusExpr 1048
+---------------- header -----------------+----- entity -----+-- time --+
```

## Parsing Channel PARS
Expand Down Expand Up @@ -243,33 +243,42 @@ of strategies used to locate a RPG/Java program.
## Analytics Channel ANALYTICS
The analytics channel provides processed collected during the whole execution cycle. Information yield by this channel are derived by a LoggingContext where logging metadata are stored.

Statement time analytics provide information about which statements were called during the program execution, how much time was spent totally to execute that statement and how many times each one was hit.
Statement time analytics provide information about which statements and expressions were called during the program execution, how much time was spent totally to execute them and how many times each one was hit.

```
12:25:35.612 ANALYTICS MUTE10_10 STMT TIME DISPLAY 204us 1
12:25:35.613 ANALYTICS MUTE10_10 STMT TIME EXSR 450369us 1
12:25:35.613 ANALYTICS MUTE10_10 STMT TIME SET 146us 1
12:25:35.613 ANALYTICS MUTE10_10 STMT TIME TIME 3026us 2
12:25:35.613 ANALYTICS MUTE10_10 STMT TIME DO 441230us 1
12:25:35.613 ANALYTICS MUTE10_10 STMT TIME EVAL 3857us 2
12:25:35.613 ANALYTICS MUTE10_10 STMT TIME ZADD 204600us 200000
12:25:35.613 ANALYTICS MUTE10_10 STMT TIME SUBDUR 601us 1
+------------- header ------------+--------+-------------------+----- analytics ------+
12:42:44.064 ANALYTICS T40_A20_P17 STMT TIME MOVEA 7325609 200000
12:42:44.065 ANALYTICS T40_A20_P17 STMT TIME EXSR 7458565 200000
12:42:44.065 ANALYTICS T40_A20_P17 STMT TIME DO 9345378 2
12:42:44.065 ANALYTICS T40_A20_P17 STMT TIME EVAL 4306 4
12:42:44.065 ANALYTICS T40_A20_P17 EXPR TIME DataRefExpr 47253 200002
12:42:44.066 ANALYTICS T40_A20_P17 EXPR TIME %TRIMR 92 1
12:42:44.066 ANALYTICS T40_A20_P17 EXPR TIME StringLiteral 2 1
12:42:44.066 ANALYTICS T40_A20_P17 EXPR TIME ArrayAccessExpr 42 1
12:42:44.066 ANALYTICS T40_A20_P17 EXPR TIME IntLiteral 7 3
12:42:44.066 ANALYTICS T40_A20_P17 PARS TIME RPGLOAD 3027 1
12:42:44.066 ANALYTICS T40_A20_P17 PARS TIME PARSER 93840 1
12:42:44.066 ANALYTICS T40_A20_P17 PARS TIME CHKPTREE 514 1
12:42:44.066 ANALYTICS T40_A20_P17 PARS TIME AST 174884 1
12:42:44.066 ANALYTICS T40_A20_P17 PARS TIME LEXER 54853 1
12:42:44.066 ANALYTICS T40_A20_P17 PARS TIME PREPROP 5115 1
12:42:44.066 ANALYTICS T40_A20_P17 PARS TIME RCONTEXT 161939 1
+---------- header ---------+------ prog ------+---------- entity ---------+- time -+- hit -+
```

Symbol table analytics provide the same information of the statement analytics but referred to symbol table actions rather than statement execution.
Symbol table analytics provide the same information of other analytics but referred to symbol table actions rather than statement execution.
```
12:25:35.613 ANALYTICS MUTE10_10 SYMTBL TIME INIT 21429us 1
12:25:35.613 ANALYTICS MUTE10_10 SYMTBL TIME LOAD 131us 1
+------------- header ------------+--------+------------------+-- analytics --+
12:42:44.066 ANALYTICS T40_A20_P17 SYMTBL TIME INIT 14457 1
12:42:44.066 ANALYTICS T40_A20_P17 SYMTBL TIME LOAD 1051 1
12:42:44.066 ANALYTICS T40_A20_P17 SYMTBL TIME GET 44663 400011
12:42:44.066 ANALYTICS T40_A20_P17 SYMTBL TIME SET 164870 100007
+---------- header ---------+------ prog ------+---------- entity ---------+- time -+- hit -+
```

There are also other analytics which provide their own informational data. Here are some examples:
```
12:25:35.615 ANALYTICS MUTE10_10 EXPR TIME 56195us 1300104
12:25:35.615 ANALYTICS MUTE10_10 LOG TIME 56177us 1300109
12:25:35.615 ANALYTICS MUTE10_10 PROGRAM TIME 1220526us
+------------- header ------------+--------+---------------+--- analytics --+
12:42:44.070 ANALYTICS LOG TIME 61886 1900103
12:42:44.070 ANALYTICS INTERPRETATION TIME 4706819 1
+---------- header ---------+------ prog ------+---------- entity ---------+- time -+- hit -+
```

## Error Channel ERR
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ object MainExecutionContext {
// default values in case jariko is not called from the command line program
private val context: ThreadLocal<Context> by lazy { ThreadLocal<Context>() }
private val noContextIdProvider: AtomicInteger by lazy { AtomicInteger() }
private val noContextAttributes: MutableMap<String, Any> by lazy { mutableMapOf<String, Any>() }
private val noContextAttributes: MutableMap<String, Any> by lazy { mutableMapOf() }
private val noConfiguration: Configuration by lazy { Configuration() }
private val noProgramStack: Stack<RpgProgram> by lazy { Stack<RpgProgram>() }
private val noParsingProgramStack: Stack<ParsingProgram> by lazy { Stack<ParsingProgram>() }
Expand Down Expand Up @@ -175,6 +175,11 @@ object MainExecutionContext {
context.get()?.renderLog(renderer)
}

/**
* Checks if logging is enabled
*/
val isLoggingEnabled get() = context.get()?.isLoggingEnabled ?: false

/***
* Get DB File Factory
*/
Expand Down Expand Up @@ -211,14 +216,15 @@ data class Context(
DBFileFactory(it.nativeAccessConfig)
}
) {

private val logHandlers: MutableList<InterpreterLogHandler> by lazy {
systemInterface.getAllLogHandlers()
}

fun renderLog(renderer: LazyLogEntry) {
logHandlers.renderLog(renderer)
}

val isLoggingEnabled get() = logHandlers.isNotEmpty()
}

data class ParsingProgram(val name: String) {
Expand Down
Loading