Add tab-completion for DFHack commands

[PeridexisErrant]

Because what modern shell doesn't have tab completion?

From here.

[warmist]

Linux and macosx should work okay (even in lua repl) i'm not sure windows console can do that. Though "command-prompt" plugin should be easy to modify (maybe even i could do that).

[PeridexisErrant]

cmd.exe does have tab completion for filenames, if not for commands. If it's impossible or impractical that's fine, I'm just passing feedback along :)

[expwnent]

It's definitely a good idea if it's possible. If they want tab-completion within commands/scripts that's probably asking too much.

[warmist]

i imagine you could call the script with --tab-complete and use return string as suggestion.

[TheBloke]

I had this thought myself the other day. And not only tab completion would be useful: backwards searching, command line editing hotkeys, advanced history editing commands (eg repeat last command without its final argument), would all be nice.

My first thought would be to Incorporate the GNU readline library into DFHack. This is the CLI library used by Bash and other shells, and many other programs in the GNU/Linux world that provide their own CLI/shell. It is very sophisticated.

It is GPL licensed and the ZLib license that Core uses is compatible with GPL, so it should be possible to use readline with DFHack? So long as the CLI of DFHack is also under ZLib - I assume it's counted as part of Core?

I've also had a quick look round and found three other alternatives to readline:

1. Editline Based on NetBSD's CLI library.
   • BSD licensed.
   • Active: last release 03/15.
2. libeditline A C++ compiled fork of Editline
   • BSD licensed.
   • Abandoned since 2011.
3. Tecla. An independent project based on tcsh's CLI.
   • X11 licensed.
   • Probably active: last release 11/14.

If you guys think the incorporation of such a library, as a replacement for the main CLI loop and the current basic history handling, would be a good thing, I will look into seeing how easy/hard it would be to add one of the above to DFHack.

In addition to tab completion, I for one would love to be able to easily edit the command line (eg using EMACS keys such as Control-U for backwards copy/delete, Control-Y for paste), reverse-search through the history (Control-R by default in readline), and use special edit commands like Escape-underscore to repeat the last command without its last argument :)

[warmist]

I think most of this does not apply to windows console...

[TheBloke]

You may well be right, but I'm not sure yet.

My understanding was that cmd.exe could run any command line app, and then if that app has in-built history, search, tab completion etc, it would work fine.

Put another way: if readline compiles OK on Windows (which it definitely does with Cygwin and Mingw; VC++ is another question, and one possible stumbling block), why wouldn't a readline-enable command line app work normally from the standard Windows console?

What would be the difference between the current DFHack console, with its self-written command line and history, versus an alternative that used readline (or an alternative such as editline)? What would make one work and the other not?

Or is the issue that CMD.exe doesn't support some of the key combinations that would be required? Control-U/K/Y, Control-R, Alt-various, etc?

I'm no expert on Windows development or the Windows console so it might well be impossible, but I would be interested to understand why. I will keep researching it in the meantime.

Or an alternative is to supply a different command line for Windows, such that Windows DFHack does not use CMD.exe but instead is bundled with something like:

• Clink, a readline-enabled CMD.exe replacement for Windows with source on Github.
• Mingw Editline, a similar project using the BSD Editline I mentioned in my OP.

That may well be too much work or too much bloat of course. Then again, reinventing the wheel with self-written tab completion would also be some amount of work, and the end result would not be as capable as using an existing powerful library.

To be clear, I'm not hugely invested in the idea of readline or any such library coming to DFHack. It would improve my life using DFHack but only a small amount - it's not like I type commands into DFHack all day like I do an OS shell :) So it's a lot of work, and/or more work than just writing some simple tab completion code, then fair enough. But I thought I'd raise it as a possibility as if it isn't a much more work it would seem to me a good way to give a much more powerful console using a tried and tested library.

[TheBloke]

One example that might be relevant: I just ran command line Vim from Windows 10's cmd.exe, and it works fine - colours, screen display, keyboard shortcuts and all.

Admittedly Windows 10's cmd.exe is improved in various ways since earlier versions of Windows. I'll have to try it later on Windows 7.

But I'm hopeful that it shows in principle that more advanced command line apps can run from cmd.exe as long as they implement everything themselves.

The main issue would be compiling readline (or another suitable library) on Windows. But maybe it could be a drop-in binary library like how the Lua DLL is provided, meaning it wouldn't need to be compilable with VC and could be compiled with Mingw or whatever, which definitely works.

[PeridexisErrant]

Wow, sudden interest! As it happens I've recently been looking at the licenses of DFHack dependancies, so I have some info that may be useful:

• the current system is not self-written; it's a minimal library called linenoise
• linenoise lists "completion" as a feature, and reading the source shows a function to register tab-completions
• it's FreeBSD licensed (ie 2-clause), so GPL-compatible but probably replaced if we use GNU stuff

So I think it's possible to get tab-completion for available commands working without a new library.

If the real limit is Windows support, clink looks pretty cool (and like total overkill, but if it works...). Good alias support from clink would also cover #701, if we went that way.

[lethosor]

It's not exactly linenoise - it's based on linenoise but has been heavily modified for thread-safety. It's also only used on OS X and Linux.

[PeridexisErrant]

Huh. In that case I guess bundling something else for Windows might make sense.

[lethosor]

It would be great if we could find something that supports ANSI escape sequences, so we can use the same thing on all platforms. The MSYS console is a little better than the standard Windows command prompt (which I think is identical to what DFHack uses), but still doesn't play well with linenoise (and presumably the standard readline distribution - there are forks with Windows support, but I'm not sure if they're kept up-to-date).

[kazimuth]

What about ANSICON? It looks fairly lightweight and adds support for ANSI stuff to cmd.exe.

[PeridexisErrant]

Closing this issue - low priority, only affects Windows, and difficult to fix.

[lethosor]

It affects other platforms too - it's just more difficult to get working on Windows.

[lethosor]

I'm working on adding readline support for Linux and macOS, which would theoretically allow tab-completion there (although my main goal is to allow multi-line editing, which the Windows console has had for years, and the ability to use more readline commands). I'm not sure how to go about adding tab-completion on Windows, though.

[myk002]

the in-game gui/launcher interface now provides tab completion, history search, and line editing cursor movement. It's not the console, but at least something has it now.