Spring refactor

[joveeater]

Another wall of text incoming. I want to lay out the wider plan of what I want to do with actors. Briefly, phase 1 is the game object memory stuff that has a WIP up. Phase 2 I'm going to wrangle the codegen stuff we're using for LUA into doing automatic persistence and json config for actors and hopefully LUA created actors, but more than the features, it's about exploring the codegen stuff and ensuring we have what we'll need later.

After that it's straight on to character stats and abilities. That means effects are getting an overhaul to be passive bags of stats, any active abilities (including reactive and constantly active) are going to become actors instead. Effects use a fixed list of properties that can be extended in json, they have simple types (bool, int, float) and a selection of aggregation rules. Checks for the presence of mutations/cbms etc should instead check for one of the effect properties. Those systems will start to just manage the lifetimes of effects and actors, at least as far as modifying the player's abilities goes. The effects section of the ui is becoming the effects/actors section, they can both indistinguishably optionally add themselves to that display. Though effects use strings and actors a method.

I want to be careful about where we add actors and what we use them for, for now I'll probably add a new type for each game object but the eventual goal is to only have 2 main types. Activity actors are things that creatures are currently focused on like player activities and monster attacks etc. They're still actors and will have all the autogen/deferral stuff etc, but they have the same basic interface they do now.

The new type of actors live directly on a game object which can have any number of actors. They have a big interface full of things like on_smash, on_consume, on_pickup etc, but different types of game objects only support the methods that make sense for them. Actors can go on multiple types of object but trying to place an actor that implements a method on an object that doesn't support it is an error unless you annotate the actor's method with an optional macro. This should hopefully sometimes make the common request of "how do I make furniture into vehicle part etc" as simple as using the same actor. There'll only be a nod to this in the short term, but longer term I want to support swapping actors around as much as possible.

Actors have before_smash etc methods as well. These are predicates, returning false should prevent the smash from happening, but they also aren't deferred or safe. You shouldn't be actively doing anything from a before method, though you can easily defer stuff then as we'll see later. Also with that same rule there will be methods to do things like add stuff to all the various menus etc. At some point I'd like to do a generic game object interface with methods to get things like position or name etc and I'd also like to look at separating "effect X costs resource Y."

But how will this all work under the hood? Code generation is the answer to the boilerplate and type checking issues, but internally we have a 3rd type of actor called free actors. Free actors are a functor (i.e. callable class) which takes no arguments and has serialize and deserialize. Deferred methods all come with a way to wrap them and their arguments up into a free actor.

Free actors are like persistent callbacks. They are very powerful and we should probably support things like wrapping up free functions but mostly they're there to gloss over all the different methods of bigger actors and let us run them all from the same list. That said the temporary "to be executed this turn" queue doesn't actually store free actors, just 0 argument functions of any sort. It's only longer delays (e.g. a list of actors that want to run every turn) where we might want to save the list that actually need free actors. So before_ code etc can just throw a lambda into the temporary list to do active things safely.

Hopefully this won't mean too much rejigging of the turn order. I do want to move map shifts to the end of the turn. At some point I want to get rid of the coordinate system shift (or at least move it to the big overmap blocks) but even just changing where the reality bubble is and what's loaded is a problem and having that change "between turns" lets us limit the impact as an edge case. Other than that though, after each major step like the player taking an action or vehicles moving, we just process the temporary list, including chain reactions.

Anyway, that's all a long way off, once I'm done with character abilities I'm gonna regroup on the whole thing. How much will actually be moved to actors right now remains to be seen, the aim there is more to get the json interface looking nice while also being backwards compatible. At which point our interfaces will be nicely in place and we can worry about fancy stuff like free actors. That said, converting stuff over is probably more pressing than fancy stuff, we'll see how it all pans out.

At this point the code shows phase 1 (at least for items) pretty well and it's actually turned out to be relatively painless considering. There's still plenty to be done and I'm gonna follow this post up with a more detailed description of phase 2.

[joveeater]

Oh there's one snarl left in phase 1 when it comes to character ids. They're getting replaced with safe_reference which is just a creature reference under the hood. Their always loaded data is it's own little thing that uses safe references as keys and because it's always storing a reference itself npcs always have an ID in json. There'll be a bit of munging to translate old ids in there as well.

There's another point I forgot to mention when talking about mutations/cbms. Effects and actors have a property that prevents them from being saved normally. This is set by things like mutations and then the mutation will give you its effect again when you load. This means things that get changed get updated on load. Also while effects and actors aren't game objects, they can still use safe references but they have their own referencing that leans on the idea that they're part of a game object and usually the same object so we can just use a reference to that if needed at all. It's only systems like cbms/mutations that are managing effects/actors that should be using that though, other things just check the resulting properties etc. That's all phase 3 though.

Phase 2 is mostly wrangling codegen. I'm gonna use the same codegen stuff as LUA, I haven't looked into the tech of it at all yet but the intent is that you can inherit from an actor class and use simple annotation macros. There'll be one that lets you annotate the class with the actor's name, one for properties that allows them to be configured from json. One that enables auto serialization and one that disables it for specific properties. More than that though I want to be sure that we can get all of the boilerplate to have nice free actors that'll be well named in the json later.

The real meat of the problem is LUA actors and it's here that I'd very much like @olanti-p 's input. What I'm thinking is that as the first step in loading mods, before any json definitions are read, we give each mod's lua a chance to define new actors, this way we know what to expect when we load in mod definitions or the save game. I haven't worked much with newer versions of lua so I don't know the best way but what I'm thinking is that during this step you'd build a lua object for each new actor type.

You can add methods to the object at this point and if they coincide with the actor interface they'll be hooked up but if not we still make them available from lua kinda like private methods. There'll also be access to json configuration types. So something like obj.foo = config_int() will allow foo: 6 when you're adding this actor to an object in the json, with some care towards required and defaults etc.

Once you're actually dealing with live actors it's a little different. No more adding random methods, all of their properties should be serializable, maybe excepting a special cache property, but they should of course allow you to call all the methods defined during startup on them. Hopefully we can get our ability to serialize types to the stage where we can serialize the actors that modders want to build.

On the C++ side there'll be a 2 autogen'd LUA actor classes of each type that're wrapping all this up. One is for actor definitions, it wraps up the objects created during the first step and one is created for each new actor type. The other is for instances of lua actors (and is actually an actor) and just stores their instance data and a reference to which type they are. Naming actors will be very important. Their names need to be descriptive of where the code for them can be found, such that they can safely live across saves and avoid naming conflicts as much as possible.

For backwards compatibility C++ actors will keep their names and to support auto naming actors later we can use C++'s rule that you shouldn't start a type name with __. LUA actors and free actors will use names that start with _LUA and FREE internally. These names are used in save game json. Mods can only use short names, of which actors can have none, which makes free actors inaccessible from mods, one for c++ actors where it matches their normal name, or two for lua actors, one that matches their normal name, another with [mod_name] at the start. If there's a naming conflict C++ actors win but debugmsg you not to name your mod actors the same as an engine actor. LUA you just get your own mod's version of that actor if you have one or an error otherwise. This all gets resolved away when we read in the mod json, once we're up and running it's long names only.

I'm really not sure how much of the LUA side of this we'll get done right away, even when I get to it it's more about finding my way around the codegen and working out the stuff to start reading in the actor's interfaces. I want to get some basic interfaces in place and then move on to phase 3 rather than get too deep into implementation now.

[olanti-p]

Well, as was mentioned on Discord the Lua PR (#2216) in its current form does not use codegen. It should definitely simplify creation of new bindings though.

Regarding the implementation of actor "types" created on load and actor "live instances" used for the game world - this can be easily done with metatables. Before load, the Lua scripts would define a "type" table and fill it with methods, and then "live" tables would just use the "type" tables for their __index metamethod.

Example snippet

-- Global actor types table
actor_types = {}
-- Contains methods and constants, defined before load
actor_types["my_actor_type"] = {
  type_name = "my_actor_type",
  init = function(self)
    self.name = "unnamed"
  end,
  set_name = function(self, new_name)
    self.name = new_name
  end,
  print_hello = function(self)
    print("Hello, "..self.name.."!")
  end,
  get_type = function(self)
    return type_name
  end
}

-- Creates live actor of given type
local make_new = function(type_name)
  local ret = {}
  local actor_type_indexer = {
    __index = actor_types[type_name]
  }
  setmetatable( ret, actor_type_indexer )
  ret:init()
  return ret
end

local live_actor_bn = make_new("my_actor_type")
local live_actor_dda = make_new("my_actor_type")
live_actor_bn:set_name("BN")
live_actor_dda:set_name("DDA")
live_actor_bn:print_hello()   -- Prints "Hello, BN!"
live_actor_dda:print_hello()   -- Prints "Hello, DDA!"
print(live_actor_bn:get_type())   -- Prints "my_actor_type"
print(live_actor_dda:get_type())   -- Prints "my_actor_type"

Example of creating and using live actors on C++ side with sol2

Example snippet

Lua

-- Global actor types table
actor_types = {}
-- Contains methods and constants, defined before load
actor_types["my_actor_type"] = {
  type_name = "my_actor_type",
  init = function(self)
    self.name = "unnamed"
  end,
  set_name = function(self, new_name)
    self.name = new_name
  end,
  print_hello = function(self)
    print("Hello, "..self.name.."!")
  end,
  get_type = function(self)
    return self.type_name
  end
}

C++

struct lua_actor {
    sol::table t;
};

template<typename Ret, typename ...Args>
Ret call_actor_method( lua_actor &actor_instance, const std::string& method_name, Args&&... args )
{
    // Have to query the metatable here because accessing with [] bypasses objects's `__index` metamethod
    sol::protected_function func = actor_instance.t[sol::metatable_key][method_name];
    // TODO: catch errors
    return func( actor_instance.t, std::forward<Args>( args )... );
}

lua_actor make_instance( sol::state& lua, const std::string& actor_type_name )
{
    // actor_types table has been populated with actor "type" tables by mods
    sol::table actor_type = lua.globals()["actor_types"][actor_type_name];
    sol::table actor_instance = lua.create_table();
    actor_instance[sol::metatable_key] = actor_type;
    lua_actor ret{ actor_instance  };
    call_actor_method( ret, "init" );
    return ret;
}

// Creating actor type from C++, as an alternative to running the script
void create_actor_type( sol::state& lua )
{
    sol::table type = lua.create_table();
    type["type_name"] = "my_cpp_actor";
    type["init"] = [](sol::table self) {
        self["name"] = "unnamed";
    };
    type["set_name"] = [](sol::table self, std::string new_name) {
        self["name"] = new_name;
    };
    type["print_hello"] = [](sol::table self) {
        std::string name = self["name"];
        debugmsg( "Hello, %s!", name );
    };
    type["get_type"] = [](sol::table self) -> std::string {
        return self[sol::metatable_key]["type_name"];
    };
    lua.globals()["actor_types"]["my_cpp_actor"] = type;
}

void do_stuff( sol::state& lua )
{
    // The Lua script that defines the type must have been already executed
    lua_actor actor_bn = make_instance( lua, "my_actor_type" );
    lua_actor actor_dda = make_instance( lua, "my_actor_type" );
    call_actor_method( actor_bn, "set_name", "BN" );
    call_actor_method( actor_dda, "set_name", "DDA" );
    call_actor_method( actor_bn, "print_hello" );  // Prints "Hello, BN!"
    call_actor_method( actor_dda, "print_hello" );  // Prints "Hello, DDA!"
    std::string t_bn = call_actor_method<std::string>( actor_bn, "get_type" );  // Returns "my_actor_type"
    std::string t_dda = call_actor_method<std::string>( actor_dda, "get_type" );  // Returns "my_actor_type"
}

Hopefully we can get our ability to serialize types to the stage where we can serialize the actors that modders want to build.

There's actually already a function that recursively goes over given sol::table and serializes in to JSON, and a counterpart that builds a sol::table out of JSON. I haven't hooked up the usertypes (C++ classes exposed to Lua) yet, but it should be just a matter of having the codegen register a pair of de-/serialization functions from classes' existing serialize/deserialize functions.

These names are used in save game json. Mods can only use short names, of which actors can have none, which makes free actors inaccessible from mods, one for c++ actors where it matches their normal name, or two for lua actors, one that matches their normal name, another with [mod_name] at the start.

I'm not sure I follow here, but I think it'd be simpler to just have mandatory prefixes for everything.
Say, core:aim_activity for activity actor defined in C++, bn:lighter_iuse for Lua item use actor defined in the "bn" core content pack, magiclysm:weak_healing_spell for some custom actor provided by Lua mod. This way, we don't run the risk of confusing which mod the actor comes from when it comes to mod interop.

[scarf005]

another possible refactoring for this spring:

• Pass by value refactor #2467
• Use std::string_view for printing functions #2459
• add convenience methods for json serialization
• make code terser using aggregate initialization

[scarf005]

resolved by #2250