Logging and Privacy

Cloud-based Diagnostic Logging and Privacy

This software gathers diagnostic data and posts it to our logging service in the cloud

Some users may find this undesirable for a variety of reasons. We will explain here how the process works and how to reconfigure and/or disable it completely. You, the end user, are in complete control. If you wish to disable or modify logging, please see the Privacy section below.

We generally require cloud logging to be enabled for alpha and beta builds as a condition of early access. Users of release builds may leave it enabled or disable it as they prefer.

Cloud-based logging was introduced in version 4.0.0 of the ASCOM driver. None of this affects the firmware or other applications that do not use the ASCOM driver.

Rationale

In the past we have found it difficult to obtain diagnostic data from non-technical end users. Each time, we had to:

• request the log file from the user
• explain to the user where to find the logs
• explain how to reproduce a problem and capture it in the logs
• explain how to control the logging verbosity
• get the correct log file back to us in the correct format
• try to work out what happened in a sterile environment.

That was just an error-prone and time consuming process and what we ended up with was a text file with fairly limited information. We wanted:

• a way to capture relevant data along with log messages;
• to require no technical knowledge from the user;
• and ideally no action at all from the user, other than to report the issue.

So we developed a Semantic Logging solution that sends rich log data to a dedicated Linux server in the cloud. The driver sends log output to this service, and lets us view and query it for up to 90 days. This not only lets as find past errors, but also to view live log data with contextual information. For example, if we log an exception (error) then we get a stack trace in out log output, that shows us the path through the code to the point where the error happend:

This makes it much easier for us to work effectively with users to resulve problems.

Technical Description

Logging Technology

We use the following technologies as part of our logging strategy:

• TA.Utilities - our own internal Logging abstractions and utilities
• NLog - a mature .NET logging platform which supports semantic logging. The use of semantic logging allows us to capture data as well as log messages.
• NLog.Targets.Seq - a log renderer plugin for NLog that sends logging data to a remote server in JSON format.
• Seq a Linux based log aggregation and querying service that runs on our cloud server.

The end result is a web page where we can view live log data:

One of the log events is shown expanded, and you can see that it contains quite a bit of rich data, including information about the system state, any variables that we chose to include in the log entry, and the name of the machine and logged in user where the log entry was generated.

We also get a dashboard view that gives us a high level view of what's happening:

We can control the logging verbosity at the server, so that clients only send the minimum required amount of log data. During pre-release alpha- and beta-test phases, or when troubleshooting issues, we can increase the verbosity to gather more data. At other times, we may log only exceptions.

Log data is kept for up to 90 days and then permanently deleted. The data is used only for diagnostic purposes.

Logging is performed very efficiently and you should not notice any effect on performance. Log data is transmitted asynchronously so it does not hold up the program. Entries that cannot be sent within a reasonable time are simply discarded and will not "block" the program. Even if we take the server completely offline, the driver will continue to work and log data will simply be discarded.

The NexDome driver additionally writes text log files in the logs directory within your user home directory. Typically, that will be C:\Users\{your-login-name}\logs. NexDome log files all begin with TA.Nexdome.Server- followed by the date, followed by your Windows login name, followed by the machine name. For example: TA.Nexdome.Server-2020-10-24-TYCO_Tim-TYCO.log. These files can be viewed with any text editor, Visual Studio Code works very well but Notepad will do.

Privacy

You, the end user, are in complete control. Here we explain how to modify the logging configuration so that you can alter what information is sent or disable logging altogether.

NLog configuration

We use a logging framework called NLog, which looks for a configuration file called NLog.config in the program directory. The program is typically installed in C:\Program Files (x86)\Tigra Astronomy\ASCOM Server for NexDome and that's where you'll find NLog.config.

Completely Disable Logging

You can completely disable all logging by renaming or deleting the NLog.config file. It's that simple.

Controlling Individual Log Targets

NLog has the ability to write to multiple targets. The configuration that we ship with the driver writes to three log targets by default:

1. Our cloud log collector (Seq)
2. To a text file on your hard drive
3. To the "debug" output stream, which lets you view log data live using a program such as DebugView (SysInternals) or LogFusion (Binary Fortress).

You can see this configuration in the <Rules> section of the NLog.config file:

  <rules>
    <logger enabled="true" name="*.ServedComClassLocator" final="true" />
    <logger enabled="true" name="*.Dome" minlevel="Trace" writeTo="DebugView" />
    <logger enabled="true" name="*" minlevel="Trace" writeTo="DebugView" />
    <logger enabled="true" name="*" minlevel="Trace" writeTo="LogFile" />
    <!-- Delete the following line to disable logging to the cloud -->
    <logger name="*" minlevel="Trace" writeTo="seq" />
  </rules>

You can control each type of log output by deleting the rule that enables it. So, deleting the line containing writeTo="seq" disables cloud logging.

You can find detailed information about NLog configuration at the NLog Project web site.