Investigate memory leaks

[Bonifatius94]

Issue:

• executing the chesslib excessively for several hours causes memory leaks
• find out if the C code causes memory leaks

[Bonifatius94]

Using this code in TensorFlow dataset generators (e.g. for training DeepQ algorithms) causes huge memory leaks within minutes. The TensorFlow logic does not seem to be the reason for the issue, so it almost certainly has to be related to memory leaks in this package.

[Bonifatius94]

There have been fixes for some memory leaks, shipped with recent releases.
Check whether the memory leaks are still a thing for the fixed versions.

[Bonifatius94]

Added another fix. RAM allocation seems to be stable now.
-> do further investigations in performance tests

[Bonifatius94]

memory leaks still remain, even after assigning all numpy arrays the data ownership

[Bonifatius94]

the issue seems to be that the API functions taking numpy arrays are not decrementing the reference counters after they are finished, so the python engine accidentially thinks the objects are still in use at some function
-> check if reference counter decrement helps solving this problem
-> add reference count assertions to the unit tests

[Bonifatius94]

Use following code for performance testing:

import sys
import chesslib
import numpy as np


def choose_action(board, drawing_side, last_draw):
    poss_actions = chesslib.GenerateDraws(board, drawing_side, last_draw, True)
    return np.random.choice(poss_actions)


def play_single_game():

    # init cache variables
    last_draw = chesslib.ChessDraw_Null
    board = chesslib.ChessBoard_StartFormation()
    drawing_side = chesslib.ChessColor_White

    # init game state
    state = chesslib.GameState_None
    game_over_states = [chesslib.GameState_Checkmate, chesslib.GameState_Tie]

    # log the start formation to console
    # print('start of game')
    # print(chesslib.VisualizeBoard(board))

    # continue until the game is over
    while state not in game_over_states:

        # choose an action and apply it to the board
        next_action = choose_action(board, drawing_side, last_draw)
        board = chesslib.ApplyDraw(board, next_action)
        
        #print('board refs:', sys.getrefcount(board))

        # update the cache variables
        last_draw = next_action
        drawing_side = 1 - drawing_side # alternate between white and black side
        state = chesslib.GameState(board, last_draw)

        # log the game progress to console
        # print()
        # print(chesslib.VisualizeDraw(next_action), f'state={state}')
        # print(chesslib.VisualizeBoard(board))

    # log the game's outcome to console
    winning_side = "black" if drawing_side == chesslib.ChessColor_White else "white"
    # print('tie' if state == chesslib.GameState_Tie else f'{winning_side} player won')


def main():

    # play 1 mio games for performance testing
    for i in range(1000000):
        print(f'playing game # {i+1}')
        play_single_game()


if __name__ == '__main__':
    main()

[Bonifatius94]

chesslib is still leaking ... but a lot less, so the effort had actually a positive effect
maybe keep this issue still in mind and try to fix it in the future