Documentation + separate routes and exports into their own file

functions/src/admin/readme.md

@@ -1,15 +1,20 @@
-# Firebase Admin
+# Admin
 
-`admin.ts` handles centrally initializing the Firebase Admin to access various different services across different files / functions. 
+[admin.ts](./admin.ts) handles centrally initializing the Firebase Admin to access various different services across different files / functions. 
 
-### Usage
+### Admin.ts
 
-When needing to use additional services from firebase instantiate them in `admin.ts` and import them where needed. Also use this place to pass in custom arguments or different configuration parameters. 
+When needing to use additional services from firebase instantiate them in [admin.ts](./admin.ts) and import them where needed. Also use this place to pass in custom arguments or different configuration parameters. 
 
+### Auth0.ts
+
+This service handles performing OAuth exchanges to get access tokens. Also used by the event checkin service to add callback urls to Auth0.
 ### Notes
 
  - The `service-account.json` file is not passed in as a paramter to `admin.initializeApp()` because the backend is deployed on Firebase Functions and those variables are automatically accessible as environment variables. If the backend is shifted elsewhere then those variables will need to be passed in.
  - If you choose write any standalone scripts store them in this folder.
+ - When creating services that perform OAuth client-credential exchanges create new files for them here
+ - Ensure that an `async get_auth_token` is present any services that perform authentication. Reuse the same token per API transaction instead of instantiating a new one for each function call.
 
 ### Questions
 

functions/src/application/readme.md

@@ -4,3 +4,47 @@ This code drives the majority of the backend responsible for powering the ACM Po
 
 ### Portal Functions
 
+```
+// fill at later point in time
+```
+### Typeform Functions
+
+The typeform functions section is very mature and is responsible for all interactions with the third party service. 
+
+##### Open Webhook
+
+The `typeform_webhook` function present in [typeform.ts](./typeform.ts) is responsible for receiving all incoming webhook requests from typeform. When creating a new typeform make sure to add in the following url as a webhook: `https://us-central1-acm-core.cloudfunctions.net/challenge/typeform`.
+
+This function will extract all the information within the typeform response body and save only the relevant fields in the following format. An array of `qa` type objects represents the final state of the object that is persisted in Cloud Firestore. This format makes it easy to quickly retrieve typeform responses and perform additional manipulation with Firestore Triggers.
+
+```
+export interface qa {
+  question: string;
+  answer: string;
+  type: string;
+}
+```
+
+Hidden fields are also extracted from Typeform submissions and saved in the same format. In this situation the `question` will be the hidden field key and the `answer` will be the hidden field value. 
+
+##### Confirmation Emails
+
+One essential part to ensuring that members who submit forms to ACM are happy and aware that their submission was received is to send them confirmation emails. While typeform natively supports sending confirmation emails it does not support the ability to customize and design them to match the ACM Brand Guidelines. In [typeform.ts](./typeform.ts) we have the `send_confirmation` firestore trigger. This function gets run anytime a new typeform document gets saved in Cloud Firestore. 
+
+This functions queries the `typeform_meta` firestore collection with the `typeform_id` to see whether there exists a confirmation email that should be sent. If the exists information regarding the Sendgrid template / sender information then this firestore trigger will call the Sendgrid API to send out that specific email. Adding a new configuration can be done through the officer utility [Sendgrid x Typeform](https://survey.acmutd.co/email). 
+
+##### Custom Actions
+
+Sometimes we want to perform a custom action based on the submission of a typeform. These are typically the internal utilities made for the ACM officer to refine workflows. All custom action functions are saved in [Custom Actions](../custom). These are also Firestore Triggers and get executed anytime a new document is created. The following functions have custom triggers:
+
+ - [Vanity Form](https://survey.acmutd.co/vanity)
+ - [Sendgrid Form](https://survey.acmutd.co/email)
+ - [Event Form](https://survey.acmutd.co/event)
+
+### Questions
+
+Sometimes you may have additional questions. If the answer was not found in this readme please feel free to reach out to the [Director of Development](mailto:[email protected]) for _ACM_
+
+We request that you be as detailed as possible in your questions, doubts or concerns to ensure that we can be of maximum assistance. Thank you!
+
+![ACM Development](https://www.acmutd.co/brand/Development/Banners/light_dark_background.png)

functions/src/application/typeform.ts

@@ -194,7 +194,6 @@ export const custom_form_actions = functions.firestore
           return;
       }
     } catch (error) {
-      console.log(error);
       logger.log({
         ...error,
         message: "Error occured in custom typeform function",

functions/src/divisions/divisions.ts

@@ -1,3 +1,7 @@
+/**
+ * @deprecated file
+ */
+
 import { firestore } from "../admin/admin";
 import { Response, Request } from "express";
 import * as Sentry from "@sentry/node";

functions/src/events/events.ts

@@ -1,3 +1,7 @@
+/**
+ * @deprecated file
+ */
+
 import { firestore } from "../admin/admin";
 import { Response, Request } from "express";
 import * as Sentry from "@sentry/node";

functions/src/express_configs/readme.md

@@ -46,4 +46,12 @@ Unauthenticated requests can be sent to `/portal`. This is useful for situations
 https://us-central1-acm-core.cloudfunctions.net/portal
 ```
 
-The endpoints are deployed under `/portal`
+The endpoints are deployed under `/portal`
+
+### Questions
+
+Sometimes you may have additional questions. If the answer was not found in this readme please feel free to reach out to the [Director of Development](mailto:[email protected]) for _ACM_
+
+We request that you be as detailed as possible in your questions, doubts or concerns to ensure that we can be of maximum assistance. Thank you!
+
+![ACM Development](https://www.acmutd.co/brand/Development/Banners/light_dark_background.png)

functions/src/index.ts

@@ -1,82 +1,15 @@
 /**
- * Handle express routing in this file
+ * Firebase function exports
  */
-import app_portal from "./express_configs/express_portal";
-import app_cf from "./express_configs/express_cf";
-import app_open from "./express_configs/express_open";
-
-import { Request, Response } from "express";
+import app_portal from "./routes/express_portal";
+import app_cf from "./routes/express_cf";
+import app_open from "./routes/express_open";
 
 import * as functions from "firebase-functions";
-import * as challengeFunctions from "./challenge/challenge";
-import * as hacktoberfestFunctions from "./custom/hacktoberfest";
 import * as typeformFunctions from "./application/typeform";
-import * as errorFunctions from "./services/ErrorService";
 import * as portalFunctions from "./application/portal";
-import logger, { debug_logger } from "./services/logging";
 import * as Sentry from "@sentry/node";
 
-//this will match every call made to this api.
-app_portal.all("/", (request: Request, response: Response, next) => {
-  logger.log(request);
-  //next() basically says to run the next route that matches the url
-  next();
-});
-app_cf.all("/", (request: Request, response: Response, next) => {
-  logger.log(request);
-  next();
-});
-app_open.all("/", (request: Request, response: Response, next) => {
-  next();
-});
-
-/**
- * Challenges for ACM Development
- */
-app_open.post("/tags/:tag", challengeFunctions.createTag);
-app_open.get("/tags/:tag/:token", challengeFunctions.getTag);
-app_open.patch("/tags/:tag/:token", challengeFunctions.patchTag);
-app_open.delete("/tags/:tag/:token", challengeFunctions.deleteTag);
-
-/**
- * typeform webhook
- */
-app_open.post("/typeform", typeformFunctions.typeform_webhook);
-
-/**
- * Debugging endpoints
- */
-app_open.get("/debug-sentry", errorFunctions.debug_sentry);
-app_open.get("/debug-logger", debug_logger);
-
-/**
- * htf-development retrieval
- */
-app_open.post("/htf-development", hacktoberfestFunctions.retrieve_record);
-
-/**
- * Cloudflare access protected endpoint
- */
-app_cf.get("/verify", portalFunctions.verify); //to be phased out
-app_cf.get("/verify-idp", portalFunctions.verify_idp);
-
-/**
- * The two following endpoints are duplicated across both /auth0 and /gsuite
- * This is because they have common requirements
- * Additional endpoints for separate forms will be on one or the other path
- */
-app_portal.get("/auth0/verify-idp", portalFunctions.verify_idp);
-app_portal.get("/gsuite/verify-idp", portalFunctions.verify_idp);
-
-app_portal.get("/auth0/verify", portalFunctions.verify);
-app_portal.get("/gsuite/verify", portalFunctions.verify);
-
-//all initialization requests on portal frontend for forms are get requests
-app_portal.get("/auth0/create-blank-profile", portalFunctions.create_blank_profile);
-app_portal.get("/auth0/profile", portalFunctions.get_profile);
-app_portal.get("/auth0/developer", portalFunctions.get_developer_profile);
-app_portal.get("/auth0/checkin", portalFunctions.record_event);
-
 // Automatically send uncaught exception errors to Sentry
 process.on("uncaughtException", (err) => Sentry.captureException(err));
 
@@ -87,6 +20,5 @@ export const challenge = functions.https.onRequest(app_open);
 
 // firestore triggers
 export const custom_form_actions = typeformFunctions.custom_form_actions;
-export const email_discord_mapper = hacktoberfestFunctions.mapper;
 export const create_profile = portalFunctions.create_profile;
 export const typeform_confirmation = typeformFunctions.send_confirmation;

functions/src/routes/express_cf.ts

@@ -0,0 +1,19 @@
+/**
+ * Handle express routing in this file
+ */
+import app_cf from "../express_configs/express_cf";
+import { Request, Response } from "express";
+import * as portalFunctions from "../application/portal";
+
+app_cf.all("/", (request: Request, response: Response, next: () => void) => {
+  next();
+});
+
+/**
+ * Cloudflare access protected endpoint
+ */
+app_cf.get("/verify", portalFunctions.verify); //to be phased out
+app_cf.get("/verify-idp", portalFunctions.verify_idp);
+
+// http server endpoints
+export default app_cf;

functions/src/routes/express_open.ts

@@ -0,0 +1,45 @@
+/**
+ * Handle express routing in this file
+ */
+import app_open from "../express_configs/express_open";
+
+import { Request, Response } from "express";
+
+import * as challengeFunctions from "../challenge/challenge";
+import * as hacktoberfestFunctions from "../custom/hacktoberfest";
+import * as typeformFunctions from "../application/typeform";
+import * as errorFunctions from "../services/ErrorService";
+import { debug_logger } from "../services/logging";
+
+/**
+ * Match all requests
+ */
+app_open.all("/", (request: Request, response: Response, next) => {
+  next();
+});
+
+/**
+ * Challenges for ACM Development
+ */
+app_open.post("/tags/:tag", challengeFunctions.createTag);
+app_open.get("/tags/:tag/:token", challengeFunctions.getTag);
+app_open.patch("/tags/:tag/:token", challengeFunctions.patchTag);
+app_open.delete("/tags/:tag/:token", challengeFunctions.deleteTag);
+
+/**
+ * typeform webhook
+ */
+app_open.post("/typeform", typeformFunctions.typeform_webhook);
+
+/**
+ * Debugging endpoints
+ */
+app_open.get("/debug-sentry", errorFunctions.debug_sentry);
+app_open.get("/debug-logger", debug_logger);
+
+/**
+ * htf-development retrieval
+ */
+app_open.post("/htf-development", hacktoberfestFunctions.retrieve_record);
+
+export default app_open;

functions/src/routes/express_portal.ts

@@ -0,0 +1,32 @@
+/**
+ * Handle express routing in this file
+ */
+import app_portal from "../express_configs/express_portal";
+import { Request, Response } from "express";
+import * as portalFunctions from "../application/portal";
+
+//this will match every call made to this api.
+app_portal.all("/", (request: Request, response: Response, next) => {
+  //next() basically says to run the next route that matches the url
+  next();
+});
+
+/**
+ * The two following endpoints are duplicated across both /auth0 and /gsuite
+ * This is because they have common requirements
+ * Additional endpoints for separate forms will be on one or the other path
+ */
+app_portal.get("/auth0/verify-idp", portalFunctions.verify_idp);
+app_portal.get("/gsuite/verify-idp", portalFunctions.verify_idp);
+
+app_portal.get("/auth0/verify", portalFunctions.verify);
+app_portal.get("/gsuite/verify", portalFunctions.verify);
+
+//all initialization requests on portal frontend for forms are get requests
+app_portal.get("/auth0/create-blank-profile", portalFunctions.create_blank_profile);
+app_portal.get("/auth0/profile", portalFunctions.get_profile);
+app_portal.get("/auth0/developer", portalFunctions.get_developer_profile);
+app_portal.get("/auth0/checkin", portalFunctions.record_event);
+
+// http server endpoints
+export default app_portal;

functions/src/services/readme.md

@@ -0,0 +1,78 @@
+# Logging in ACM Core
+
+Logging is an important aspect of being able to understand the sequence of events that take place in the API. Several aspects make using regular `console.log` statements not ideal in a production context. We would like to have a logging solution which is robust, scalable, queryable, triggers alerts & more. 
+
+### LogDNA
+
+The current solution used by ACM Core is using LogDNA as a third party external service. LogDNA has several premium features which are free for students and this guide will explain how to connect your LogDNA account to ACM Core. 
+
+ - [LogDNA Student](https://www.logdna.com/github-students)
+
+Set up a student account by following the steps at the link above. Once you have completed setting up an account grab your ingestion id and return to this readme.
+
+### Add to Logging Service
+
+The [logging.ts](./logging.ts) file sets up a `Logger` class that manages sending log information to everyone who has associated an ingestion id with the ACM Core project.
+
+##### Adding to Firebase Functions Config
+
+ The first thing that we will need to do is add your LogDNA Ingestion ID as an environment variable. You can add it to the firebase functions config as follows. Make sure to replace `yourname` and `your-ingestion-id` in the command below. Make sure to just use your first name and not have any spaces in the command.
+
+```
+$ firebase functions:config:set logdna.yourname=your-ingestion-id
+```
+
+After this update the `.runtimeconfig.json` file with the new environment variable. Make sure to be at the root of the repository before running this command.
+
+```
+$ firebase functions:config:get > functions/.runtimeconfig.json
+```
+
+##### Updating the Logger
+
+In [logging.ts](./logging.ts) create a new LogDNA logger instance by duplicating the line below and changing the name of the logger to be your own. Also update the environment variable being passed in to be `functions.config().logdna.yourname` that was set in the previous section.
+
+```
+const harsha_logger = logdna.createLogger(functions.config().logdna.harsha, options);
+```
+
+Next append your logger to the array of LogDNA instances within the Logger class. 
+
+```
+private logdna_loggers = [harsha_logger, YOURNAME_logger /*, insert additional loggers */];
+```
+
+That's it! Your logger will now receive all the log data from the API and can be accessed in the LogDNA cloud console.
+
+##### Using the Logger
+
+An instance of the logger class can be imported into any file as follows
+
+```
+import logger from "../services/logging";
+```
+
+Using the logger is as simple as using the single `.log()` function.
+
+```
+logger.log("hello world");
+```
+
+The logger is capable of receiving objects too which can then be queryed in the LogDNA cloud console for additional visibility.
+
+```
+logger.log({
+    ...my_object,
+    message: "logging a new message",
+});
+```
+
+You can also log entire request or response bodies when receiving an http request but this is not receommended since it makes log traversal much more challenging. There is also no need to log information like timestamps or environment context. These variables are automatically logged and tracked.
+
+### Questions
+
+Sometimes you may have additional questions. If the answer was not found in this readme please feel free to reach out to the [Director of Development](mailto:[email protected]) for _ACM_
+
+We request that you be as detailed as possible in your questions, doubts or concerns to ensure that we can be of maximum assistance. Thank you!
+
+![ACM Development](https://www.acmutd.co/brand/Development/Banners/light_dark_background.png)