Skip to content

Commit

Permalink
feat: add basic user role support to backend (#1094)
Browse files Browse the repository at this point in the history
* feat: update role enums: UserRole and ProjectRole

* feat: update DbUserRole model to use ProjectRole enum + comp key

* build: update base schema with new enum types

* fix: osm login methods to async

* feat: add basic roles: super admin & validator

* build: add migration for ProjectRole db enum type

* build: fix migrations with revert for projectrole

* build: fix enum migrations with intermediate varchar

* fix: handle invalid access tokens

* refactor: correct use of 403 http status over 401

* refactor: tidy minor code edits & lint fixes

* build: default to 4 workers on dev uvicorn container

* feat: add project_deps with get_project_by_id logic

* feat: add org_admin role to role deps

* fix: add user role to response /me for frontend

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
  • Loading branch information
spwoodcock and pre-commit-ci[bot] authored Jan 17, 2024
1 parent d4b20b1 commit f8cf65f
Show file tree
Hide file tree
Showing 14 changed files with 400 additions and 75 deletions.
125 changes: 90 additions & 35 deletions docs/About.md
Original file line number Diff line number Diff line change
@@ -1,77 +1,132 @@
![](https://github.com/hotosm/fmtm/blob/main/images/hot_logo.png?raw=true)

# Field Mapping Tasking Manager (FMTM)

## 📖 History

### How was FMTM born?

It started as Ivan's idea to build FMTM (Ivan Gayton is Senior Humanitarian Advisor at Humanitarian OpenStreetMap Team) which then became a collaborative project with the efforts of Ivan , Rob Savoye who is Senior Technical Lead at Humanitarian OpenStreetMap Team and many other members from HOT as well as volunteers interested in the project.
HOT uses ODK heavily, but most of the data never makes it into OSM because all the data processing is manual and slow, so it doesn't get done.<img align="right" width="300px" src="https://github.com/hotosm/osm-fieldwork/assets/97789856/afc791c7-6cc9-4496-aa93-ab02733f30b8"/>
Ivan Gayton(Senior Humanitarian Advisor at Humanitarian OpenStreetMap Team) heard about what Rob was working on and goes "That's the missing piece I needed!". He'd been wanting to build FMTM for years, but lacked the ability to process the data.A [webinar](https://www.youtube.com/watch?v=GiLKRZpbtrc&ab_channel=HumanitarianOpenStreetMapTeam) then took place in September 2022 that showcased the high interest from the community and the need for collaborative field mapping that really kicked off the starting point for building the Field Mapping Tasking Manager. It was Ivan who got HOT interested enough to direct some resources to his idea, so FMTM was born.

<!-- <img align="left" width="300px" src="https://github.com/hotosm/osm-fieldwork/assets/97789856/afc791c7-6cc9-4496-aa93-ab02733f30b8"/> -->

_Want to know about OSM-fieldwork project ?_ Click [here](https://github.com/hotosm/osm-fieldwork/wiki)
It started as Ivan's idea to build FMTM (Ivan Gayton is Senior Humanitarian
Advisor at Humanitarian OpenStreetMap Team) which then became a collaborative
project with the efforts of Ivan, Rob Savoye who is Senior Technical Lead at
Humanitarian OpenStreetMap Team and many other members from HOT as well as
volunteers interested in the project.

HOT uses ODK heavily, but most of the data never makes it into OSM because all
the data processing is manual and slow, so it doesn't get done.
<img align="right" width="300px" src="https://github.com/hotosm/osm-fieldwork/assets/97789856/afc791c7-6cc9-4496-aa93-ab02733f30b8"/>

Ivan Gayton(Senior Humanitarian Advisor at Humanitarian OpenStreetMap Team)
heard about what Rob was working on and goes "That's the missing piece I
needed!". He'd been wanting to build FMTM for years, but lacked the ability to
process the data.
A [webinar](https://www.youtube.com/watch?v=GiLKRZpbtrc&ab_channel=HumanitarianOpenStreetMapTeam) then took place in September 2022
that showcased the high interest from the community and the need for
collaborative field mapping that really kicked off the starting point for
building the Field Mapping Tasking Manager. It was Ivan who got HOT interested
enough to direct some resources to his idea, so FMTM was born.

**\_Want to know about OSM-fieldwork project ?**
Click [here](https://github.com/hotosm/osm-fieldwork/wiki)
<br>
<br>
<br>
<br>

## A project to provide tools for Open Mapping campaigns

The Field Mapping Tasking Manager (FMTM) is a project that aims to provide tools for coordinating field mapping activities in Open Mapping campaigns. While there are existing field mapping applications, there is a lack of efficient tools to coordinate these activities. The FMTM builds on the HOT Tasking Manager and other mapping applications to provide a more streamlined and organized process for completing mapping tasks.
The Field Mapping Tasking Manager (FMTM) is a project that aims to provide tools
for coordinating field mapping activities in Open Mapping campaigns. While
there are existing field mapping applications, there is a lack of efficient
tools to coordinate these activities. The FMTM builds on the HOT Tasking
Manager and other mapping applications to provide a more streamlined and
organized process for completing mapping tasks.

Currently, it is possible to implement a Field Mapping Tasking Manager workflow
using existing tools, but it requires significant effort and can be challenging.

Currently, it is possible to implement a Field Mapping Tasking Manager workflow using existing tools, but it requires significant effort and can be challenging. The FMTM project is developing automation features to address these challenges and make the process more accessible to users.
The FMTM project is developing automation features to address these challenges
and make the process more accessible to users.

By providing a centralized platform for organizing and managing mapping tasks, assigning them to specific users, and tracking their progress, the FMTM aims to simplify the coordination of mapping activities. The tool also provides analytics and reporting features, allowing users to gain insights into mapping campaigns and adjust their strategies accordingly.
By providing a centralized platform for organizing and managing mapping tasks,
assigning them to specific users, and tracking their progress, the FMTM aims to
simplify the coordination of mapping activities. The tool also provides
analytics and reporting features, allowing users to gain insights into mapping
campaigns and adjust their strategies accordingly.

[Background and description of the project and idea are here: please have a look at this blog if you haven't yet!](https://www.hotosm.org/updates/field-mapping-is-the-future-a-tasking-manager-workflow-using-odk/)
Background and description of the project and idea are
[here](https://www.hotosm.org/updates/field-mapping-is-the-future-a-tasking-manager-workflow-using-odk/):
please have a look at this blog if you haven't yet!

# How to contribute

The FMTM project is open source and community-driven, welcoming contributions from designers, user testers, and both front-end and back-end developers. If you're interested in getting involved, please see our [contributor guidelines](https://github.com/hotosm/fmtm/blob/main/CONTRIBUTING.md) for more information. We welcome questions and feedback, so don't hesitate to reach out to us. 👍🎉
The FMTM project is open source and community-driven, welcoming contributions
from designers, user testers, and both front-end and back-end developers. If
you're interested in getting involved, please see our
[contributor guidelines](https://github.com/hotosm/fmtm/blob/main/CONTRIBUTING.md)
for more information. We welcome questions and feedback, so don't hesitate
to reach out to us. 👍🎉

# Using OpenDataKit's Select From Map feature

OpenDataKit's Select From Map feature is a useful tool for field mappers to collect data in a well-structured questionnaire format. The tool was incorporated into ODK in mid-2022 and allows mappers to select an object from a map, view its existing attributes, and fill out a form with new information and attributes.

To prepare map files for ODK, inspiration is taken from the HOT Tasking Manager, which allows remote mappers to choose well-defined small "task" areas, ensuring full coverage of the project area and no unintended duplication of tasks. For example, a mapper can approach a building, select that building from a map view within ODK on their mobile phone, and add the opening hours, number of floors, construction material, or any number of useful attributes in a well-structured questionnaire format

<!-- <img src="https://github.com/hotosm/fmtm/blob/main/images/ODK_Select_one_from_file_map_screenshot.jpg?raw=true" width=800 height= 800> -->

To prepare the appropriate map files for ODK, we are taking our inspiration from the [HOT Tasking Manager](https://tasks.hotosm.org/), which allows remote mappers to choose well-defined small "task" areas, ensuring full coverage of the project area and no unintended duplication of tasks.

<!-- <img src="https://github.com/hotosm/fmtm/blob/main/images/HOT_TM_task_selection_screenshot.jpg?raw=true" width=800 height= 800> -->
OpenDataKit's Select From Map feature is a useful tool for field mappers to
collect data in a well-structured questionnaire format. The tool was
incorporated into ODK in mid-2022 and allows mappers to select an object from a
map, view its existing attributes, and fill out a form with new information
and attributes.

To prepare map files for ODK, inspiration is taken from the HOT Tasking Manager,
which allows remote mappers to choose well-defined small "task" areas, ensuring
full coverage of the project area and no unintended duplication of tasks. For
example, a mapper can approach a building, select that building from a map
view within ODK on their mobile phone, and add the opening hours, number of
floors, construction material, or any number of useful attributes in a
well-structured questionnaire format

To prepare the appropriate map files for ODK, we are taking our inspiration from
the [HOT Tasking Manager](https://tasks.hotosm.org/), which allows remote
mappers to choose well-defined small "task" areas, ensuring full coverage
of the project area and no unintended duplication of tasks.

# Users

There are three main user roles for using ODK's Select From Map feature: campaign managers, field mappers, and validators.
There are three main user roles for using ODK's Select From Map feature:
campaign managers, field mappers, and validators.

## Campaign managers

Campaign managers select an Area of Interest (AOI) and organize field mappers to go out and collect data. They need to:
Campaign managers select an Area of Interest (AOI) and organize field mappers
to go out and collect data. They need to:

<img align="right" width="400px" src="https://github.com/hotosm/fmtm/assets/97789856/9343a4bc-462c-44af-af93-8a67907837b3"/>

- Select an AOI polygon by creating a GeoJSON or by tracing a polygon in a Web map
- Choose a task division scheme (number of features or area per task, and possible variations on what features to use as the preferred splitting lines)
- Select an AOI polygon by creating a GeoJSON or by tracing a polygon
in a Web map
- Choose a task division scheme (number of features or area per task,
and possible variations on what features to use as the preferred splitting lines)
- Provide specific instructions and guidance for field mappers on the project.
- Provide a URL to a mobile-friendly Web page where field mappers can, from their mobile phone, select a task that is not already "checked out" (or possibly simply allocate areas to the field mappers).
- See the status of tasks (open, "checked out", completed but not validated, requires to rework, validated, etc) in the Web browser on their computer
- Provide a URL to a mobile-friendly Web page where field mappers can, from
their mobile phone, select a task that is not already "checked out"
(or possibly simply allocate areas to the field mappers).
- See the status of tasks (open, "checked out", completed but not validated,
requires to rework, validated, etc) in the Web browser on their computer

## Field mappers

Field mappers select (or are allocated) individual tasks within a project AOI and use ODK Collect to gather data in those areas. They need to:
Field mappers select (or are allocated) individual tasks within a project AOI
and use ODK Collect to gather data in those areas. They need to:

- Visit a mobile-friendly Web page where they can see available tasks on a map
- Choose an area and launch ODK Collect with the form corresponding to their allocated area pre-loaded
- Choose an area and launch ODK Collect with the form corresponding to their
allocated area pre-loaded

## Validators

Validators review the data collected by field mappers and assess its quality. If the data is good, the validators merge the portion of the data that belongs in OpenStreetMap to OSM. If it requires more work, the validators either fix it themselves (for minor stuff like spelling or capitalization mistakes that don't seem to be systematic) or inform the field mappers that they need to fix it. They need to:
Validators review the data collected by field mappers and assess its quality.
If the data is good, the validators merge the portion of the data that
belongs in OpenStreetMap to OSM. If it requires more work, the validators
either fix it themselves (for minor stuff like spelling or capitalization
mistakes that don't seem to be systematic) or inform the field mappers
that they need to fix it. They need to:

- Access completed data sets of "submissions" as Comma Separated Values and/or OSM XML so that they can review them.
- Access completed data sets of "submissions" as Comma Separated Values
and/or OSM XML so that they can review them.
- Mark areas as validated or requiring rework
- Communicate with field mappers if rework is necessary
- Merge good-quality data into OSM (probably from JOSM).
Expand Down
2 changes: 1 addition & 1 deletion src/backend/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ RUN pip install --user --upgrade --no-warn-script-location \
&& rm -r /opt/python
CMD ["python", "-m", "debugpy", "--listen", "0.0.0.0:5678", \
"-m", "uvicorn", "app.main:api", \
"--host", "0.0.0.0", "--port", "8000", \
"--host", "0.0.0.0", "--port", "8000", "--workers", "4", \
"--reload", "--log-level", "critical", "--no-access-log"]


Expand Down
4 changes: 4 additions & 0 deletions src/backend/app/auth/auth_routes.py
Original file line number Diff line number Diff line change
Expand Up @@ -162,9 +162,13 @@ async def my_data(
)
db.add(db_user)
db.commit()
# Append role
user_data["role"] = db_user.role
else:
if user_data.get("img_url"):
user.profile_img = user_data["img_url"]
db.commit()
# Append role
user_data["role"] = user.role

return JSONResponse(content={"user_data": user_data}, status_code=200)
27 changes: 22 additions & 5 deletions src/backend/app/auth/osm.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,22 +24,29 @@
from fastapi import Header, HTTPException, Request
from loguru import logger as log
from osm_login_python.core import Auth
from pydantic import BaseModel
from pydantic import BaseModel, ConfigDict

from app.config import settings
from app.models.enums import UserRole

if settings.DEBUG:
# Required as callback url is http during dev
os.environ["OAUTHLIB_INSECURE_TRANSPORT"] = "1"


class AuthUser(BaseModel):
"""The user model returned from OSM OAuth2."""

model_config = ConfigDict(use_enum_values=True)

id: int
username: str
img_url: Optional[str]
role: Optional[UserRole]


def init_osm_auth():
async def init_osm_auth():
"""Initialise Auth object from osm-login-python."""
return Auth(
osm_url=settings.OSM_URL,
client_id=settings.OSM_CLIENT_ID,
Expand All @@ -50,8 +57,11 @@ def init_osm_auth():
)


def login_required(request: Request, access_token: str = Header(None)):
osm_auth = init_osm_auth()
async def login_required(
request: Request, access_token: str = Header(None)
) -> AuthUser:
"""Dependency to inject into endpoints requiring login."""
osm_auth = await init_osm_auth()

# Attempt extract from cookie if access token not passed
if not access_token:
Expand All @@ -62,4 +72,11 @@ def login_required(request: Request, access_token: str = Header(None)):
if not access_token:
raise HTTPException(status_code=401, detail="No access token provided")

return osm_auth.deserialize_access_token(access_token)
try:
osm_user = osm_auth.deserialize_access_token(access_token)
except ValueError as e:
log.error(e)
log.error("Failed to deserialise access token")
raise HTTPException(status_code=401, detail="Access token not valid") from e

return osm_user
133 changes: 133 additions & 0 deletions src/backend/app/auth/roles.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
# Copyright (c) 2022, 2023 Humanitarian OpenStreetMap Team
#
# This file is part of FMTM.
#
# FMTM is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# FMTM is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with FMTM. If not, see <https:#www.gnu.org/licenses/>.
#

"""User roles authorisation Depends methods.
These methods use FastAPI Depends for dependency injection
and always return an AuthUser object in a standard format.
"""

from fastapi import Depends, HTTPException
from loguru import logger as log
from sqlalchemy.orm import Session

from app.auth.osm import AuthUser, login_required
from app.db.database import get_db
from app.db.db_models import DbProject, DbUser, DbUserRoles
from app.models.enums import HTTPStatus, ProjectRole, UserRole
from app.projects.project_deps import get_project_by_id


async def get_uid(user_data: AuthUser) -> int:
"""Extract user id from returned OSM user."""
if user_id := user_data.get("id"):
return user_id
else:
log.error(f"Failed to get user id from auth object: {user_data}")
raise HTTPException(
status_code=HTTPStatus.UNAUTHORIZED,
detail="Auth failed. No user id present",
)


async def super_admin(
db: Session = Depends(get_db),
user_data: AuthUser = Depends(login_required),
) -> AuthUser:
"""Super admin role, with access to all endpoints."""
user_id = await get_uid(user_data)

match = db.query(DbUser).filter_by(id=user_id, role=UserRole.ADMIN).first()

if not match:
log.error(f"User ID {user_id} requested an admin endpoint, but is not admin")
raise HTTPException(
status_code=HTTPStatus.FORBIDDEN, detail="User must be an administrator"
)

return user_data


async def org_admin(
project: DbProject = Depends(get_project_by_id),
org_id: int = None,
db: Session = Depends(get_db),
user_data: AuthUser = Depends(login_required),
) -> AuthUser:
"""Organization admin with full permission for projects in an organization."""
user_id = await get_uid(user_data)

org_admin = (
db.query(DbUserRoles)
.filter_by(user_id=user_id, role=ProjectRole.ORGANIZATION_ADMIN)
.first()
)

if not org_admin:
log.error(f"User ID {user_id} is not an admin for any organization")
raise HTTPException(
status_code=HTTPStatus.FORBIDDEN,
detail="User must be an organization admin",
)

matched_project = db.query(DbProject).filter_by(id=org_admin.project_id).first()
matched_org_id = matched_project.organisation_id

if (
org_id
and matched_org_id == org_id
or project
and matched_org_id == project.organisation_id
):
return user_data

log.error(f"User ID {user_id} is not an organization admin for id {org_id}")
raise HTTPException(
status_code=HTTPStatus.FORBIDDEN, detail="User is not an organization admin"
)


async def validator(
project_id: int,
db: Session = Depends(get_db),
user_data: AuthUser = Depends(login_required),
) -> AuthUser:
"""A validator for a specific project."""
user_id = await get_uid(user_data)

match = (
db.query(DbUserRoles).filter_by(user_id=user_id, project_id=project_id).first()
)

if not match:
log.error(f"User ID {user_id} has no access to project ID {project_id}")
raise HTTPException(
status_code=HTTPStatus.FORBIDDEN, detail="User has no access to project"
)

if match.role.value < ProjectRole.VALIDATOR.value:
log.error(
f"User ID {user_id} does not have validator permission"
f"for project ID {project_id}"
)
raise HTTPException(
status_code=HTTPStatus.FORBIDDEN,
detail="User is not a validator for this project",
)

return user_data
Loading

0 comments on commit f8cf65f

Please sign in to comment.