Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FEAT: User Auth on Diagnostics Page #441

Merged
merged 32 commits into from
Jan 8, 2023
Merged
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
8c36beb
Remove sync message
Jan 1, 2023
ba7b461
Add login form and auth view
Jan 1, 2023
c602abe
Fix login error message bug. Add logout button to diag page.
Jan 2, 2023
7e01e6b
Use blocks to promote reuse in HTML templates
Jan 2, 2023
d64b5b8
Update button colours, add password change form
Jan 2, 2023
57922cb
Add password strength test
Jan 2, 2023
8aa0c08
Resolve SonarQube code smell
Jan 2, 2023
dd43dd6
Refactor auth file writing
Jan 2, 2023
da51621
Remove deprecated HTML tags
Jan 2, 2023
c05f3d8
Fix typo
Jan 2, 2023
70857a9
Merge pull request #440 from robputt/FEAT-user-auth
robputt Jan 2, 2023
4b30054
Fix failing tests since changes
Jan 2, 2023
803f23b
Style updates
Jan 4, 2023
05bb898
Refactor password write function
Jan 4, 2023
3cf29e0
Instantiate empty alembic migrations
Jan 5, 2023
4b7d5a4
Update requirements
Jan 5, 2023
fefeb41
Add DB and migrations
Jan 5, 2023
ff8bda8
Move WSGI entrypoint to avoid imports error generating migrations dur…
Jan 5, 2023
f5e1707
Merge branch 'master' into FEAT-user-auth
robputt Jan 5, 2023
66a8fbe
Only include the Python module hw_diag in Flake8, avoids failing auto…
Jan 5, 2023
12f968e
Fix up Dockerfile
Jan 5, 2023
ed8d39f
Run DB migrations on application start
Jan 5, 2023
a44fa13
Move from password file to sqlite3 DB
Jan 5, 2023
77f82ab
Move migrations runner out of app.py
Jan 5, 2023
6eea6f8
Add auth failures table
Jan 5, 2023
e663479
Lock login form if too many failures in last 10 minutes
Jan 5, 2023
a87c5d9
Fix tests
Jan 5, 2023
87d818c
Add password reset page
Jan 7, 2023
1916b46
Enhance password reset experience
Jan 7, 2023
cc0fb26
Check Iinternal IP and pre shared api key
Jan 7, 2023
720a994
Utilise JINJA2 template for unauthenticated pages
Jan 7, 2023
7a37d07
Add notice for devices without a button
Jan 7, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion .github/workflows/python-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,14 @@ jobs:
- name: Lint with flake8
run: |
pip install flake8
flake8 . --count --max-complexity=10 --statistics
flake8 hw_diag/. --count --max-complexity=10 --statistics
- name: Move migrations to expected location
run: |
mkdir /opt/migrations
cp -r migrations /opt/migrations/.
cp -r alembic.ini /opt/migrations/.
sudo mkdir /var/data
sudo chmod 777 /var/data
- name: Unit tests
run: |
pip install pytest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we add pytest-cov to test-requirememts and add

--cov=hw_diag --cov=bigquery --cov-fail-under=70

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally I would agree, but I think it's out of scope of this PR. Let's raise this in a separate PR if there is a policy change regarding coverage.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, we have that on other repos such as config https://github.com/NebraLtd/hm-config/blob/c580b108ecacedae69b3b0c2954704646d2118a0/.github/workflows/python-tests.yml#L22

But yeah can put in a follow up ticket if that's easier

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow up ticket #445

Expand Down
8 changes: 5 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,6 @@ RUN mkdir /tmp/build
COPY ./ /tmp/build
WORKDIR /tmp/build



RUN \
install_packages \
build-essential \
Expand Down Expand Up @@ -70,8 +68,12 @@ COPY --from=builder /usr/sbin/QFirehose /usr/sbin/QFirehose
# copy firmware files
COPY --from=builder /tmp/build/quectel /quectel

# copy db migration files
COPY --from=builder /tmp/build/migrations /opt/migrations/migrations
COPY --from=builder /tmp/build/alembic.ini /opt/migrations/alembic.ini

# Add python dependencies to PYTHONPATH
ENV PYTHONPATH="${PYTHON_DEPENDENCIES_DIR}:${PYTHONPATH}"
ENV PATH="${PYTHON_DEPENDENCIES_DIR}/bin:${PATH}"

ENTRYPOINT ["gunicorn", "--bind", "0.0.0.0:5000", "--timeout", "300", "hw_diag:wsgi_app"]
ENTRYPOINT ["gunicorn", "--bind", "0.0.0.0:5000", "--timeout", "300", "hw_diag.wsgi:wsgi_app"]
4 changes: 3 additions & 1 deletion MANIFEST.in
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
include hw_diag/templates/diagnostics_page.html
include hw_diag/templates/template.html
include hw_diag/templates/password_change_form.html
include hw_diag/templates/login_form.html
include hw_diag/templates/diagnostics_page_light_miner.html
include hw_diag/static/css/bootstrap.min.css
include hw_diag/static/fonts/*.ttf
Expand Down
105 changes: 105 additions & 0 deletions alembic.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# A generic, single database configuration.

[alembic]
# path to migration scripts
script_location = migrations

# template used to generate migration file names; The default value is %%(rev)s_%%(slug)s
# Uncomment the line below if you want the files to be prepended with date and time
# see https://alembic.sqlalchemy.org/en/latest/tutorial.html#editing-the-ini-file
# for all available tokens
# file_template = %%(year)d_%%(month).2d_%%(day).2d_%%(hour).2d%%(minute).2d-%%(rev)s_%%(slug)s

# sys.path path, will be prepended to sys.path if present.
# defaults to the current working directory.
prepend_sys_path = .

# timezone to use when rendering the date within the migration file
# as well as the filename.
# If specified, requires the python-dateutil library that can be
# installed by adding `alembic[tz]` to the pip requirements
# string value is passed to dateutil.tz.gettz()
# leave blank for localtime
# timezone =

# max length of characters to apply to the
# "slug" field
# truncate_slug_length = 40

# set to 'true' to run the environment during
# the 'revision' command, regardless of autogenerate
# revision_environment = false

# set to 'true' to allow .pyc and .pyo files without
# a source .py file to be detected as revisions in the
# versions/ directory
# sourceless = false

# version location specification; This defaults
# to migrations/versions. When using multiple version
# directories, initial revisions must be specified with --version-path.
# The path separator used here should be the separator specified by "version_path_separator" below.
# version_locations = %(here)s/bar:%(here)s/bat:migrations/versions

# version path separator; As mentioned above, this is the character used to split
# version_locations. The default within new alembic.ini files is "os", which uses os.pathsep.
# If this key is omitted entirely, it falls back to the legacy behavior of splitting on spaces and/or commas.
# Valid values for version_path_separator are:
#
# version_path_separator = :
# version_path_separator = ;
# version_path_separator = space
version_path_separator = os # Use os.pathsep. Default configuration used for new projects.

# the output encoding used when revision files
# are written from script.py.mako
# output_encoding = utf-8

sqlalchemy.url = sqlite:////var/data/hm_diag.db


[post_write_hooks]
# post_write_hooks defines scripts or Python functions that are run
# on newly generated revision scripts. See the documentation for further
# detail and examples

# format using "black" - use the console_scripts runner, against the "black" entrypoint
# hooks = black
# black.type = console_scripts
# black.entrypoint = black
# black.options = -l 79 REVISION_SCRIPT_FILENAME

# Logging configuration
[loggers]
keys = root,sqlalchemy,alembic

[handlers]
keys = console

[formatters]
keys = generic

[logger_root]
level = WARN
handlers = console
qualname =

[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine

[logger_alembic]
level = INFO
handlers =
qualname = alembic

[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic

[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S
3 changes: 0 additions & 3 deletions hw_diag/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +0,0 @@
from hw_diag.app import get_app

wsgi_app = get_app(__name__)
27 changes: 27 additions & 0 deletions hw_diag/app.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
import logging
import os
import uuid
import traceback
from datetime import datetime
from functools import partial
from datetime import timedelta

from flask import Flask
from flask import g
from flask_apscheduler import APScheduler

from hm_pyhelper.logger import get_logger
Expand All @@ -16,7 +18,12 @@
from hw_diag.utilities.network_watchdog import NetworkWatchdog
from hw_diag.utilities.sentry import init_sentry
from hw_diag.views.diagnostics import DIAGNOSTICS
from hw_diag.views.auth import AUTH
from hw_diag.utilities.quectel import ensure_quectel_health
from hw_diag.database.config import DB_URL
from hw_diag.database import get_db_session
from hw_diag.database.migrations import run_migrations


SENTRY_DSN = os.getenv('SENTRY_DIAG')
DIAGNOSTICS_VERSION = os.getenv('DIAGNOSTICS_VERSION')
Expand Down Expand Up @@ -97,12 +104,32 @@ def init_scheduled_tasks(app) -> None:


def get_app(name):
# Run database migrations on start...
run_migrations('/opt/migrations/migrations', DB_URL)

app = Flask(name)
cache.init_app(app)
init_scheduled_tasks(app)

# Setup DB Session
@app.before_request
def pre_request():
g.db = get_db_session()

@app.after_request
def post_request(resp):
try:
g.db.close()
except Exception:
pass
return resp

# Use a random UUID for session key, this will change each time the app
# starts, so with reboot / update etc... users will need to reauthenticate.
app.secret_key = str(uuid.uuid4())

# Register Blueprints
app.register_blueprint(DIAGNOSTICS)
app.register_blueprint(AUTH)

return app
26 changes: 26 additions & 0 deletions hw_diag/database/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker

from hw_diag.database.config import DB_URL


BASE = declarative_base()


def get_db_engine(debug=False):
engine = create_engine(DB_URL, echo=debug)
return engine


def get_db_session(debug=False):
sessmaker = sessionmaker(bind=get_db_engine(debug))
session = sessmaker()
return session


# These imports are down here to prevent cyclic imports
# they are not used in this file but are required for
# alembic to include tables in revision generation.
from hw_diag.database.models.auth import AuthKeyValue # noqa: E402,F401
from hw_diag.database.models.auth import AuthFailure # noqa: E402,F401
1 change: 1 addition & 0 deletions hw_diag/database/config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
DB_URL = 'sqlite:////var/data/hm_diag.db'
11 changes: 11 additions & 0 deletions hw_diag/database/migrations.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
import logging
from alembic.config import Config
from alembic import command


def run_migrations(script_location, dsn):
logging.info('Running DB migrations in %r on %r', script_location, dsn)
alembic_cfg = Config()
alembic_cfg.set_main_option('script_location', script_location)
alembic_cfg.set_main_option('sqlalchemy.url', dsn)
command.upgrade(alembic_cfg, 'head')
Empty file.
34 changes: 34 additions & 0 deletions hw_diag/database/models/auth.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
from sqlalchemy import Column
from sqlalchemy import String
from sqlalchemy import DateTime


from hw_diag.database import BASE


class AuthKeyValue(BASE):
__tablename__ = 'auth_kv'

key = Column(
String(60),
nullable=False,
primary_key=True
)
value = Column(
String(250),
nullable=False
)


class AuthFailure(BASE):
__tablename__ = 'auth_failures'

dt = Column(
DateTime(),
nullable=False,
primary_key=True
)
ip = Column(
String(45),
nullable=True
)
2 changes: 1 addition & 1 deletion hw_diag/static/css/bootstrap.min.css

Large diffs are not rendered by default.

90 changes: 5 additions & 85 deletions hw_diag/templates/diagnostics_page_light_miner.html
Original file line number Diff line number Diff line change
@@ -1,77 +1,9 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta http-equiv="refresh" content="300">
<title>Nebra Hotspot Diagnostics</title>
<!-- Bootstrap core CSS -->
<link href="/static/css/bootstrap.min.css" rel="stylesheet">
<link rel="shortcut icon" href="/static/images/favicon.ico">
<meta name="theme-color" content="#03a9f4">
<meta name="robots" content="noindex, nofollow">
</head>
<body>
<header class="bg-white mb-4 d-flex justify-content-center">
<a href="#" class="p-3 text-center">
<img src="/static/images/nebra-logo.svg" height="32" alt="Nebra Logo" />
</a>
</header>
<section class="container pb-4">
{% extends 'template.html' %}

<div class="alert alert-primary alert-dismissible fade show" role="alert">
<h2 class="alert-heading">What happened to sync percentage?</h2>
<p>Helium has converted all existing hotspots into <a
href="https://blog.helium.com/light-hotspots-explained-everything-you-need-to-know-f86612f571c6"
target="_blank">light hotspots</a>. Sync percentage and other blockchain features are now handled by
validators, not hotspots. We appreciate your patience with any issues during the transition period.</p>
<a href="https://engineering.helium.com/2022/07/14/miner-hotspot-release.html" target="_blank"
class="btn btn-light">More Information</a>
<style>
.alert {
border-radius: 8px;
}
{% block title %}Diagnostics{% endblock %}

.alert-primary {
background-color: #02A8F5;
color: white;
}

.alert-primary a {
color: white;
font-weight: 600;
}

.alert-primary .btn-light {
color: #02A8F5;
border-radius: 50px;
}
</style>
</div>

{% if diagnostics.error %}
<div class="text-center pb-4 mb-4 border-bottom border-light">
<h1>Diagnostics Information</h1>
<br />
<h3>{{ diagnostics.error }}</h3>
</div>
{% else %}
<div class="text-center pb-4 mb-4 border-bottom border-light">
<h1>Diagnostics Information</h1>
<br />
<h3>
{% if not diagnostics.AN %}
Animal Name Unavailable
{% else %}
{{ diagnostics.AN }}
{% endif %}
</h3>
{% if diagnostics.PF %}
<h3 class="text-success">All Ok</h3>
{% else %}
<h3 class="text-danger">Errors Found</h3>
{% endif %}
</div>
{% block body %}
{% if not diagnostics.error %}
<h3 class="text-center mb-4">Diagnostics Breakdown</h3>
<div class="row mb-4">
<div class="col-12 col-lg-6 mb-4 mb-lg-0">
Expand Down Expand Up @@ -171,16 +103,4 @@ <h3 class="text-center mb-4">Diagnostics Breakdown</h3>
</div>
</div>
{% endif %}
<div class="text-center">
{% if diagnostics.last_updated %}
<p>Last Updated: {{ diagnostics.last_updated }}</p>
{% else %}
<p>Last Updated: Never</p>
{% endif %}
<p>To get support please visit <a href="https://nebra.io/helium-support">https://nebra.io/helium-support</a></p>
<p><a href="/json">Download Diagnostics Info for Support</a></p>
<p>&copy; Nebra LTD. 2020-{{ now.year }}<p>
</div>
</section>
</body>
</html>
{% endblock %}
Loading