MS2 – Data aggregator on frontend libraries

The website is intended to help front-end development students to compare the popularity (depicted by GitHub stars and Google Trends) of different JavaScript packages and keywords.

UX

Strategy & Scope

A selection of the major objectives and selected ways to achieve them in form of user stories and conclusions.

As a…

front-end development student...

I want to learn the best technology to foster my career. I have difficulties to decide which is the most relevant technology, because I lack the experience to evaluate the quality of software libraries. -> Basic Assumption: Depending on the experience of others

I want to use the experience of many professionals (not just one subjective opinion) to create a shortlist of libraries to research. -> GitHub Stars as metric
I want to get a feeling about the different trends of the libraries on my shortlist, because I don't want to accidentally select the technology, which will be obsolete before I get relevant experience with it. -> Google Trends timeline
I don't want to miss the library which is a perfect fit for my needs, because I did my research only on the superstar of the bunch. -> Selection and comparison by tags/keywords
I have to do research on my shortlist, to find my best fit. The evaluation of the relevance in my region is possible with Google Trends, the technical research is best done on the repositories. -> Include Links to both

Structure & Skeleton

The value for the user is mainly in the embedded Google Trends chart, and the filterable data table. So both of them are presented at the top of the page. The control elements connecting them are anchored between them.

On bigger screens, the main filter elements are placed next to the table, on mobiles they switch below the table, to prevent user-irritation (clear binding to the table, not the trends-chart) and maintain the table in a readable width.

The feature to search all the keywords present in the dataset is combined with a filter-slider, to prevent overloading the tagcloud to an unusable list (the 4,000 packages in the dataset got over 6,000 unique keywords (already normalized to lowercase)), but maintain the usability of the feature.

The about-section was positioned beneath the main data to hook the user with the most asked question ("Angular, React or Vue?"), afterwards guiding to critical thinking about the provided data and further research on package use.

The disclaimer and contacts sections are of minor value for the user, but the disclaimer and contacts should be presented on the same page, as the used data and therefore positioned at the bottom of the page.

Mobile wireframe

Desktop wireframe

Surface

As preset, I embed the well known Google brand colors (via the Google Trends chart). Without customization of inputs many browsers use blue as default color. To stay in the range but clearly differentiate from Googles content, I picked green for my main color and used a gradient to blue from Google and the browsers for my gradient.

Back to top

Features

Existing Features

Google Trends chart

On the first load of the page the actual comparison which led to the project idea is generated to promote the feature.
If a problem with browser settings (privacy, cookies - see known bugs/features left to implement) appears, the user is informed, and a workaround is shown.
A button to open Google Trends (preloaded with selection) in a new tab is presented, for individual research on users' selection.
The chart shows the trends of the worldwide search terms (the package names selected by the user) of the past 5 years. The user can evaluate the popularity of the different packages in relative terms based on it.

Package table

On the first load of the page the most popular packages (based on GitHub stars provided by cdnjs) are listed with a link to their project page, their actual GitHub stars and the keywords, used to tag the packages.
A pagination is used to maintain the overview for users and prevent a list of thousands on small screens. Two buttons let the user turn the pages.
A checkbox allows the user to select a package for GoogleTrends.
As soon as packages are selected, a user can remove them by clicking on the generated button labeled with the name of the package, remove all by clicking a reset button or reload the Google Trends chart by clicking the "Show me the trends!"-Button.
If the user tries to select more than the intended 3 packages, a modal informs the user to just select up to 3 packages.

Data Counter

The data counter below the chart displays the total records, the sample size and contains a button to reset filters.

Text Search

A set of freetext inputs enables users to search by the package name, the owner and a specific keyword by text entry.

Range row chart

A row chart differentiates the selection by popularity based on GitHub stars. A click on a row filters on the selected range (multiple ranges selectable).
defined segments
- more than 100,000 stars
- 10,000 to 100,000 stars
- 1,000 to 10,000 stars
- 100 to 1,000 stars
- 10 to 100 stars
- 1 to 10 stars
- zero stars
- stars not provided
The segments were chosen this way to fulfill the different criteria of users.
- compare unicorns like Facebook's React to Google's Angular
- compare niche packages like plugins for specialised use-cases

Multiple keyword selector

The multiple keyword selector lists all keywords used to tag packages in the selection, normalized to lowercase.
The thousands of packages in the unfiltered dataset contain a multiple of almost 1,5 unique (lowercase) keywords. To maintain a processable amount of useful keywords, a range slider filters the keywords based on appearance.
Checkboxes filter the selection to contain one of the selected keywords. The selected keywords are not filtered the slider.

Features left to Implement

Add a Backend to prevent loading of full dataset resources on page load and speed up TTI
Prevent layout shifting due to embed/chart generation
Customization of checkboxes and range-input for uniform cross-browser look.
Dynamic data table pagination, depending on viewport size/user input (Now: Table size 10 packages hardcoded)
- Always show selected packages
Add a widget with all active filters (for onclick removal) (Now: Find the specific chart/filter or reset all)
Customize hover-charts tooltips
Google Trends
- Find alternative way to implement Google Trends (Now: Cross-Site-Origin Error if no-track / cookies missing/ blocked); Google seems to set x-frame-options: sameorigin on their embedded response, if Googles cookies are missing. Reference
- Reverse engineer their whole embed API to make all parameters accessible for users (high risk to break, if API change)
- Research about categories, should there be one preselected, which one (exclude things like Nike React etc)
  - cat=31 coding? - https://trends.google.com/trends/explore?cat=31&geo=US&q=Vue
Keyword features
- Build a dataset of keyword-counts and
  - filter via range checkboxes/chart (analog GitHub star partials) or two inputs-slider (Now: slider)
  - set default value of keyword cloud based on selection (Now: 50 packages hardcoded)
- Alternative display of keywords, tagcloud by popularity (Now: alphabetical list)
- Show packages which contain all selected keywords (Now: packages which contain one of selected keywords)
- Rewrite filter on text search widget to allow multiple keywords
- Highlight keywords matching the keyword text filter
Write automated tests

Back to top

Technologies

cdnjs API

for the dataset
Documentation

Google Trends

for the embedded trends chart
No documentation available via Google APIs Explorer and search on Google developers
A post (dated 2014) on an abandoned blog (last post 2016) provides some insights, but uses at least an outdated entry- point. Hacking the Google Trends API, Techslides

HTML

for the main pages

CSS

for everything styling related

Bootstrap

for the responsive layout and modal

FontAwesome

for link symbols

Google Font Lato

for Lato for a clean readable impression

JavaScript

jQuery

as dependency of Bootstrap modal

For Charting and Data handling

Jasmine

for tests on load process

Git / GitHub

for version control
for deployment

gitpod

as IDE

Pycharm

as IDE, after gitpod was unavailable to frequently

code institute gitpod template

as a starter for the gitpod environment

code institute readme template

as a starter for the readme.md

coolors

for the color scheme

Back to top

Testing

Validators

Validators were used by "copy and pasting" the code into validators.

Strg+A
Strg+C
Strg+V

Validation was used on regular basis while developing and before submission.

HTML Validator

index.html

The index.html was validated by HTML validator and passed without issues.

specrunner.html

The specrunner.html was validated by HTML validator and returned a warning. The criticised "type="application/javascript"" was removed and afterwards it passed without errors.

404.html

The 404.html was validated by HTML validator and passed without issues.

CSS Validator

The styles.css was validated by jigsaw validator and passed without issues.

JS Validator

script.js

The script.js was validated by JSHint.

As it returned warnings, the following options were set;

esversion: 8 (to use "async function"; coverage per caniuse.com: 92,69%, assumption: front-end devs will use up-to-date browsers)
globals: d3, crossfilter, dc, trends, $, console (first ones imported via index.html script tags, console just error log)

No warnings remained.

spec.js

The spec.js was validated by JSHint.

As it returned warnings, the following options were set;

esversion: 8 (see script.js)
globals like in script js, added Jasmine specifics (describe, it, expect) and cdnjsFullList, cdnjsProcessedList ( declared in script.js)

No warnings remained.

Lighthouse

The Chrome Lighthouse tool was used to generate reports.

Performance

The Performance issue is due to:

The big dataset to load for basic functionality
- API call (about 200 kB)
- building crossfilter dimensions
  - A possible way to improve the behaviour could be a custom backend, serving just the data needed or , but this is beyond the scope of this project.

the call to Google Trends
- with disabled disk cache: More than 1.2 MB of 1.9 MB transferred
- only embedded version available
- undocumented API

I don't know how to circumvent or improve this issue.

Accessibility

The Accessibility score of 97 was improved to 98, by changing the heading level of the social links (h5 to h3). The missing 2 points depend on color contrast on links and buttons.

Tests with a comparison of bold, color: black, original and text-stroke (2px black), resulted in the decision to stay with the original style for best legibility.

Best Practices

The Best Practices score of 93 was improved to 100, by adding "rel="noreferrer"" to outgoing links.

Manual testing

The Browsers Chrome(v89.0.4389.82) and Firefox (v86.0) were used for testing. The deployed version of the page was tested.

A console warning derived from a removed d3 color scheme, supported by dc for backward compatibility, was removed by selecting another default color scheme (script.js line 113).
A console warning derived from the DataCount widget. Contrary to error message the crossfilter method was already used.
- the dc.filterAll() call triggers warning
  - the logger information traces down to applyFilters() in base-mixin.js (line 987), using this.dimension(); this is not easily changed by the author and was dismissed as out of scope of the project.
    - No Issue filed, due to intentional rewrite for clarity Reference
On verbose logging level Chrome reports a violation due to non-passive event listener to a scroll blocking "mousewheel" event. Probably a problem with d3 / dc js brush implementation. It was noticed, but not handled.
- Reference
Depending on browser configuration, Google Trends embedded frame is not rendered. Google seems to set x-frame-options: sameorigin on their embedded response, if Googles cookies are missing.
- Reference
- To inform the user, a custom error message is generated. As the browser cannot interfere with elements inside an iframe, the app uses a defined style attribute (provided by the embedded frame, if present) as indicator for the working feature.
All htmls were examined on different widths and responsive behavior was observed.
- 280px, 320px (as representation for mobiles)
- Bootstrap breakpoint
  - 575px, 576px
  - 767px, 768px
  - 991px, 992px
  - 1199px, 1200px
- 3000px (as representation for high resolution displays - 4k, 5k)
Google Trends in new tab button changed from google.de... to google.com.
The index.html was tabbed through.
- Popularity row chart was not tabable; .keyboardAccessible(true) added to fix.

Chrome exclusive, deployed page

Test cases

Responsiveness
- Procedure
  1. Open Homepage
  2. Left click on the page and open Dev Tools (Strg + Alt + i)
  3. Toggle device toolbar
  4. Resize with from wide to narrow
- Expected
  1. Layout aligns according to wireframes
  2. Charts redraw after resizing
  3. No Scroll bars (Exception: table, as defined in the wireframe) on page elements
  4. No breaking of the layout
- Outcome
  1. Pass
  2. Pass
  3. Pass
  4. Pass
Usability and Functionality
1. Nav
  - Procedure
    1. Scroll the page down, beginning top, observe
    2. For every nav button: click, observe
    3. Modify url (for example: "https://apometricstk.github.io/ci_ms2_aggregator/inex.html")
  - Expected
    1. Navbar sticks to top and enables user to move to distinct location on page.
    2. Every button moves the user to the according section of the page.
    3. User gets redirected to custom 404.html, where a link is available to return to index.html.
  - Outcome
    1. Pass
    2. Pass
    3. Pass
2. Filters
  1. Filtering
    - Procedure
      1. Apply a filter from every filtering element
        
        the text filters (name, owner, keyword); enter a string
        
        popularity row chart; click on a row
        
        multiple keyword range; select some checkboxes
      2. Audit expected filter behaviour
        
        Modify text input
        
        Click on the same row
        
        Click on "reset" button of multiple keyword selector
      3. Audit expected behaviour
      4. Click on "Reset All"
        
        Audit expected behaviour
    - Expected behaviour
      - Filter behaviour
        
        Text filters
        
        The filtered table column contains only elements, with the lowercased input chars in given order, without other chars in between. Before and after the sequence, other chars are allowed.
        
        name: filters first table column (package name)
        
        owner: filters second table column, hover over link symbol to compare string between the last two slashes, appearing in bottom left of the browser
        
        keyword: filters last table column (keywords provided)
        
        Adding/Removing chars to the inputs modifies filters accordingly
        
        Popularity filter
        
        The stars' column of the table contains only elements, described by the label of the row.
        
        Exception: "stars not provided": 0 is allowed
        
        The clicked row is highlighted
        
        Another click on selected row removes highlight and filter
        
        Multiple keywords range
        
        The last table column (keywords provided) contains at least one of the keywords selected by the checkboxes.
        
        If no checkbox is checked, "reset" button is disabled.
        
        Click on "reset" button removes this filter. All other filters still apply.
        
        Every adding / removing filter updates the connected elements, especially the table and the counter next to the "Reset All" button.
      - "Reset All" behaviour
        
        All filters are dropped, full dataset is displayed by the elements.
        
        "Reset All" button is disabled
    - Outcome
      - Pass
  2. Subfiltering multiple keywords
    - Procedure
      1. Moving the slider "narrow down by popularity" to the far right, observe.
      2. Moving the slider to the far left, observe.
    - Expected Behaviour
      - right: fewer keywords displayed, sentence below slider contains bold "100".
      - left: more keywords displayed, sentence below slider contains bold "1".
    - Outcome
      - Pass
3. Google Trends Comparison
  1. Selection
    - Procedure
      1. Click on up to 3 checkboxes in the table, observe
      2. Hover over appearing button, observe
      3. Click on one of the buttons, observe
      4. Try to check 4 checkboxes, observe
    - Expected behaviour
      1. Buttons with package name appear
      2. Text is crossed out
      3. The checkbox of the package named is unchecked, button disappears.
      4. A modal appears, informing you to check 1-3 boxes, just 3 boxes are checked.
    - Outcome
      1. Pass
      2. Pass
      3. Pass
      4. Pass
  2. Comparing
    - Procedure
      1. With up to 3 boxes checked, click on "Show me the trends!"
      2. With up to 3 boxes checked, click on "Google Trends in new tab"
      3. With 0 boxes checked, click on "Google Trends in new tab"
    - Expected behaviour
      1. The trends chart updates accordingly
      2. A Google Trends tab opens, preloaded with checked search-terms.
      3. A modal appears, informing you to check 1-3 boxes.
    - Outcome
      1. Pass
      2. Pass
      3. Pass
4. Data table
  - Procedure
    - "Next Page" and "Previous Page" buttons click
    - The GitHub link symbol of a button click
  - Expected behaviour
    - The table pages turn accordingly, data updates in the table and next to the buttons
    - The correct GitHub page opens in a new tab
  - Outcome
    - Pass
5. About Section
  - Procedure
    - Click every link
  - Expected behaviour
    - The link clicked opens in anew tab
  - Outcome
    - Pass
6. Contacts footer
  - Procedure
    - Click the contacts symbols
  - Expected behaviour
    - The authors contact pages opens an in new tab
  - Outcome
    - Pass
7. General
  - Procedure
    - All buttons were hovered
  - Expected behaviour
    - If not disabled, the color changes
  - Outcome
    - Pass

Automated testing

The testing site includes basic tests on correct import of font, stylesheets, libraries and the loading and processing of the data.

User-Story verification

front-end development student

I want to use the experience of many professionals (not just one subjective opinion) to create a shortlist of libraries to research. -> GitHub Stars as metric
I want to get a feeling about the different trends of the libraries on my shortlist, because I don't want to accidentally select the technology, which will be obsolete before I get relevant experience with it. -> Google Trends timeline
I don't want to miss the library which is a perfect fit for my needs, because I did my research only on the superstar of the bunch. -> Selection and comparison by tags/keywords
I have to do research on my shortlist, to find my best fit. The evaluation of the relevance in my region is possible with Google Trends, the technical research is best done on the repositories. -> Include Links to both, no further input needed.

Slack review

The project was posted to the Code Institute community Slack channel peer-code-review and to the Hackathon group-chat for different pairs of eyes.

Readme

Readme was observed on GitHub. All links were clicked.

A cross-browser issue was found

Windows 10 Firefox didn't embed a specific image of an error message, even if the links were correct. Clearing cache, reloading the page and restart of the browser didn't solve the issue.

Windows 10 Chrome embeds same image

Manjaro Firefox & Chromium embeds, other Machine Windows 10 Firefox, Edge and Chrome embeds.
Error was classified as browser-installation specific and afterwards tracked down to an Ad-blocker. Renaming of the File solved the issue.

Back to top

Deployment

The relevant files for deployment of the project are

index.html
404.html (for custom 404 error page, optional)
specrunner.html (for automated tests on loading, optional)
assets/css folder
assets/images folder (for the favicon, optional, but will return console error if omitted)
assets/js folder

The easiest way (but not the tiniest in terms of filesize - the download includes the development gitpod files) is to click on "Code" on the top of this page (top right above the file list) and on "Download zip".
Afterwards extract the zip (most operating systems include a packaging program for it, just right click on it) into the required folder (read below).

Local

For local use, download the files and the folder, put them (or extract) into the same directory and open the index.html file with your browser.

Deployed / Hosted

For deployed use, download the files and folders, put/upload/extract them into the folder advised by your hoster (e.g. "www", but it depends on the individual vendor - check back with your hoster / the documentation!).
Be cautious not to alter the file hierarchy.

with GitHub pages

If you are logged into your GitHub Account (if not: Sign up - it's free!) just click on "Fork" on the top right of this page. Afterwards in the "Settings" (top menu, right beyond the name of the repository) menu, go to "Options" (top menu item), and afterwards scroll down to "GitHub Pages", select "Master branch" as source and click "Save". To get to the deployed site afterwards, click on the link.
Further documentation

with other git instances

Clone this repository according to the documentation of your editor/git-gui (CLI: "git clone https://github.com/apometricsTK/ci_ms2_aggregator.git").

this actual deployment

The content was deployed via GitHub pages by configuring the repository according to the above walkthrough. Updates pushed to the GitHub repository will be automatically included a few minutes afterwards.

Back to top

Credits

Content

Data

Components

Browser compatibility verification with caniuse
Bootstrap components
- Layout system / grid
- Navbar
- Button
- Modal
- Card
- Accordion

Media

The Font Awesome symbols were made by Font Awesome.
The first readme screenshot was taken with ami.responsive
The color palettes screenshots were taken with coolors.co
The favicon was generated with favicon.ico and app icon generator
The wireframes were drawn with Balsamiq

Acknowledgments

My mentor Brian Macharia for his support and feedback.
My team from Code Institute hackathon March 2021 for feedback and support.
The Code Institute slack community for their support.

Back to top

Name		Name	Last commit message	Last commit date
Latest commit History 76 Commits
.theia		.theia
assets		assets
readmeAssets		readmeAssets
.gitignore		.gitignore
.gitpod.dockerfile		.gitpod.dockerfile
.gitpod.yml		.gitpod.yml
404.html		404.html
README.md		README.md
index.html		index.html
specrunner.html		specrunner.html

Kastobi/ci_ms2_aggregator

Folders and files

Latest commit

History

Repository files navigation

MS2 – Data aggregator on frontend libraries

Table of Contents

UX

Strategy & Scope

front-end development student...

Structure & Skeleton

Mobile wireframe

Desktop wireframe

Surface

Features

Existing Features

Meta

Google Trends chart

Package table

Data Counter

Text Search

Range row chart

Multiple keyword selector

Features left to Implement

Technologies

For Charting and Data handling

Git / GitHub

Testing

Validators

HTML Validator

index.html

specrunner.html

404.html

CSS Validator

JS Validator

script.js

spec.js

Lighthouse

Performance

Accessibility

Best Practices

Manual testing

Chrome exclusive, deployed page

Test cases

Automated testing

User-Story verification

front-end development student

Slack review

Readme

Deployment

Local

Deployed / Hosted

with GitHub pages

with other git instances

this actual deployment

Credits

Content

Data

Components

Media

Acknowledgments

About

Resources

Stars

Watchers

Forks

Languages