Mozilla devtools-html/debugger.html

debugger.html is a Mozilla project aimed and creating a hackable JavaScript debugger that is easy to use, while providing developers with a powerful and testable tool for the web. It was built using React and Redux. Mozilla created this tool mainly for the use in Firefox Developer Tools, however, it also works with Chrome and Node.JS. The team also invites anyone to enhance the tool to work with other environments and browsers.

Where’s the code?

The location of source code cannot be anymore obvious, it’s under src directory just under the project root folder.

Where’s the documentation?

Documentation exists under docs directory right under the project root folder. In addition to traditional documentation regarding usage of the project and project policies, Mozilla has also included a page documenting weekly updates.

How can you get involved?

One great thing about Mozilla is its strong friendly community. A great way to get involved is to join their slack team. You can get a feel for the environment of the community by idling on the channel and observing conversations. You may also notice that veteran contributors will welcome you to the team and encourage you to select a bug to work on. These folks will also extend help when you need an extra boost while tackling bugs.

In addition to joining their slack channel, the repository also includes a CONTRIBUTING.md file which contains all the details about how to contribute to the project. Common ways of contributing include reporting bugs, suggesting enhancements, writing documentation, giving talks, and organizing meetups.

Since Mozilla has a large community base and has many years of experience in opensource, the team also includes participation guidelines that they expect all members of the community to follow. This is to ensure a mutually respectful and open environment.

Finally, Mozilla has a bug bounty program. If a vulnerability is found, the community suggests reading policies regarding reporting vulnerabilities. In addition, they  recommend creating an issue on Bugzilla (Mozilla’s internal bug tracker) for security issues.

What are some interesting things you’ve learned while observing the project?

Being a first timer at an official opensource project, I’ve found the way the team has structured the issues and pull requests.

Issues

Issues generally have a screenshot (where possible). This is then followed by a short 1-2 sentence description of the issue along with examples of the problem and the expected result. Some issues will include steps to reproduce. The community has placed strong emphasis on steps to reproduce as it will help new contributors to reproduce the problem in order to tackle the bug. The issue tracker also includes a host of descriptive tags such as: in progress, not-available, difficulty:medium, enhancement, performance, bug, tracking, code health, and good first issue. In particular, “in progress” is the most useful tag as a contributor can see at a glance, which issues to skip when searching for issues to work on.

Screen Shot 2018-03-25 at 1.40.39 AM.png

The issue tracker also has a claim bot, which manages who is responsible for an issue. Contributors can reply to the issue with /claim to have the claim bot set the issue to “in progress” mode and signals to the community that he/she is currently working on the issue.

Pull requests

It’s interesting to note that the project contains a .github folder. Inside that folder, you will find files that contain templates for issues and pull requests. This is something I have not seen before, but can quickly grasp the importance of having such files for projects involving multiple contributors.

The pull requests contain the following information:
• The statement “Fixes Issue #(issue number)”
• Summary of Changes
• Test Plan (Step-by-step instructions for the reviewer to check that the fix is working)
• Screenshots of the fix (optional)

One interesting thing I’ve learned just this past week from Professor Humphrey is that GitHub has a set of keywords that can be used in a pull request title or description to allow the associated issue to close automatically upon merging. It appears Mozilla is taking full advantage of this by enforcing the “Fixes Issue…” statement in their pull requests.

Bridge-Troll project: Implement dark mode

This week, I was tasked to implement dark mode for the bridge-troll project.

Preparing for the fix

To ensure I can get updates from the original bridge-troll repo, I’ve setup the upstream remote for the repository by using the following command:

git remote add upstream https://github.com/humphd/bridge-troll.git

Trying out the app before working on it

One important thing is to ensure the app works as is and learning how it works before attempting to make changes to the app. I did a quick build to see how the app works. According to its README.md, I should be able to run debug to bypass GPS based location only, allowing me to navigate the game through double clicking the mouse.

npm run debug

However, I soon found out that the debug mode was not working. I began examining the code files, making educated guesses based on the name of the files to see where I could save the problem. I came across a file named geo.js. In this file, there was a line that enabled the debug mode. It appears the parcel.js module was having issues with parsing flags.

I bypassed the check by commenting out the if statement, enabling navigation using double mouse clicks.

The next step is to figure out where the code affecting the maps lived. Luckily, the naming convention for this project was easy to understand. Navigating to maps.js revealed the code used to generate the map.

Choosing the proper theme and icons

This application uses a node module called Leaflet JS. It is an open source JavaScript library for creating mobile friendly maps. This library contains a series of themes also known as leaflet providers.

After browsing through the series of themes, I have settled on the World Street Map theme for dark mode. The choice was chosen due to its red and yellowish tint, which reduces the amount of blue light coming from the screen at night, making the app more usable without reducing the visibility of the streets and their names.

As for the icons, the default sets were kept because they were suitable for the purpose of identifying bridges that have already been visited versus bridge that have not yet been visited. Since the roads were in a pale red color, the black locks that came default was a good choice to create contrast on the screen.

Determining sunrise and sunset

To determine the time for sunrise and sunset, I used an node module called SunCalc to detect positions of the sun relative to the time of the user’s location. This information is used to calculate the time for the end of sunrise and when sunset begins to allow automatic switching between dark and light mode.

Exporting the code to a separate module

Once our initial implementation works, our goal now is to separate the code written into its separate module for re-usability.

Creating the module and factoring out dark mode code

I began by creating a separate js file for our new module. I called the module ‘displaymode’.

At the beginning, I had trouble figuring out what parts of the code inside map.js’ init() method that could be factored out for the dark mode feature. After consulting Professor Humphrey, I realized that the new module can have its own init() method, have logic that will figure out the time of day based on given latitude and longitude arguments and expose a getMode() method to allow map.js to obtain the proper theme to render.

Exposing a setter method in DisplayMode module

In order to facilitate re-usability in other parts of the app for the future, I’ve included a setter method setMode() in the DisplayMode module, which allows the user to pass the string ‘dark’ or ‘light’ to manually set the theme of the UI.

Preparing to ship new code

Once all the code has been written, I proceeded to run

npm run eslint

Screen Shot 2018-03-18 at 3.52.03 PM.png

The results show 2 warnings and 1 error. I realized when I moved the code over from map.js into displaymode.js, I had forgotten to remove the variable that requires SunCalc node module.

I then proceeded to investigate the warnings regarding console statements in csv2json.js. However, my finding is that those 2 lines of code are related to error handling and printing results to the console. Therefore, while it is not recommended practice as outlined here, I left it alone.

The next thing I did was run npm run prettier. The purpose is to ensure the format for which my code was written conforms to the bridge-troll repository.

Creating the commit

Once linting and prettying were performed, I added all of the files using git add command and performed a git commit to upload my commit.

Pulling the latest changes

To ensure my code base is normalized against upstream, I quickly performed git pull upstream master command. I then got warnings regarding conflicts with merging with package-lock.json.

It appears that the original repository had been updated to use WebPack instead of parcel.js.

To resolve this issue, I forcefully removed package-lock.js using rm -rf package-lock.json. Then I proceeded to execute npm i to allow package-lock.json to regenerate itself; In addition, install WebPack
.
I then executed npm run debug to test the application to see if the new added code is working with the latest files pulled from upstream repository. Indeed the application loads.

To ensure night mode works, I purposefully reversed the if statement in displaymode.js to ensure the dark mode theme appears as expected.

Commit and Push the changes

Now that everything is working, I added the files and create new commit and detailed the changes I’ve made for implementing dark mode. The commit was then pushed to my own forked repository.

Creating pull request

Heading over to my own bridge-troll repository on Github, I now see a sign that says “create pull request”. I was presented with a form to fill in once I clicked the create pull request button.

I provided details regarding the issue number that I was working on and the details of the things I’ve added in the pull request.

Bloopers

One thing I forgot to do after pushing all of my commits was that I had been creating commits against my forked version’s master branch. This is very bad practice and something that I will remember to never do in the future.

I realized this when I was unable to see the create pull request link on my bridge-troll repository. To work around this issue, I created a separate branch using git checkout -b issue-6. The name issue-6 is a simple, yet descriptive name for the problem I was addressing.

Conclusion

Overall, I found this exercise to be very fun and practical way of learning the process of fixing a bug and creating a pull request. While I recall all the steps mentioned during the lesson, it was obvious that I did not recall this during the actual fix. With this exercise, I now know which areas I should pay more attention to for my future fixes.

Learning and working with ECMAScript Standardization Tests

Recently, I’ve learned about browser wars and how standardization for the web came to be. The primary focus is on ECMAScript, another name for JavaScript. If you’re interested in the history of JavaScript and how it got re-branded as ECMAScript, find out more about it here. The summary is that while Brendan Eich had created a compelling language for the web, his implementation rests with the Netscape browser, which at the time was closed source. Other browsers such as Microsoft’s Internet Explorer, desiring the same capabilities reverse engineered Brendan’s work and created their own implementation, calling it JScript to avoid trademark disputes. This means the behavior of JavaScript among browsers was not consistent. Web designers often resorted to multiple hacks to ensure cross browser compatibility. ECMA is an European organization that standardizes technology to which Brendan approached to standardize JavaScript to avoid further fragmentation and to prevent Microsoft from setting their own standards. Since the standardization of the language, software vendors and internet citizens like you and I are now able to contribute to the discussion regarding standards that should be enforced in all browsers. In order for browser vendors to ensure their browser conforms to the standard’s specifications, automated testing was created. It is open to the public such that they can also contribute by providing suggestions and feedback regarding the standard, or help write tests which browser vendors will check their implementations against. I choose to practice writing some tests for the browser vendors.

Obtaining the tests

ECMAScript has a GitHub repository, which keeps track of all tests written for ECMAScript and manages collaboration between all people working standardization. I began there.

Step 1: Read ReadMe.md

Often, I get very excited about working on an open source project that I completely skip README.md hoping to jump right into the heart of the project. The README.md on the tc39/test262 repository is very informative and is a great place to begin reading.

In particular, the runners & harnesses section is the most important. A harness in their terms means a command line software that will execute the tests written for the standard on the vendor’s browsers.

The version of harness I choose was Node.JS since I have the most experience working with it as of late.

Step 2: Install the harness

Following the commands they’ve suggested, I’ve executed the following on the terminal:

git clone https://github.com/tc39/test262.git --depth 1
cd test262
npm install -g test262-harness

Take note, the --depth 1 part of the git command is an option that specifies obtaining the latest commit only. This is useful if our purpose is to obtain the latest version of the test for conformance testing purposes.

Step 3: Execute tests

It is always a good idea to run the unit tests of a project before beginning to work on it. This way, we have an idea if there are any existing errors before introducing code of our own.

test262-harness test/**/*.js

Test results

Using Node.JS version 9.6.0, the 205 tests completed with 199 passed and 6 failed.

The problems for which Google’s V8 engine failed at seem to be related to localization and the handling of the property descriptor in its tests of the harness itself.

Background information: Node.JS is a JavaScript run-time environment built based on Google’s V8 JavaScript engine built for back end processes such as hosting a website or web service.

Step 4: Read CONTRIBUTE.md

Now that the harness is setup and tested to work, we begin by reading CONTRIBUTE.md on the repository.

After reading this document, I feel I’ve understood more about
• Tests naming conventions
• Tester file format and commenting
• How to write tests that runs on the harness

Step 5: Read the Specification

Before writing any tests, we must understand the specifications of our point of interest. The goal is to understand the implementation of Array.prototype.reverse().

This process took me a while as I had to dig deep to understand the intent of the pseudo code as well as the behavior of the abstract functions it referenced.

Step 6: Write some tests for Array.prototype.reverse()

Now that we’ve read CONTRIBUTING.md, it’s time to practice writing some tests. I’ve created array-with-primitives.js as the name of my test file. My goal was to test the following cases:

• The reversal of [] is []
• A collection of numbers is reversed correctly: [1,2] > [2,1]
• A collection of characters is reversed correctly: [‘a’,’b’] > [‘b’, ‘a’]
• A collection with mixed primitive data types should be reversed correctly: [‘a’, 1, undefined, null] > [null, undefined, 1, ‘a’]

As I experimented with writing these tests, I ran into a few problems

1) Testing the case of reversing []

The test I wrote initially was

let a = [];
let reverse = a.reverse();
assert.sameValue(
a,reverse, `#1: a = []; a.reverse(); reverse.length === 0. Actual: ${reverse}`
);

The specification noted that an object is returned and therefore I assumed a and reverse were two seperate objects to which I could use assert.sameValue() to compare.

What I’ve found over time was that this is not the case. In fact, a also gets modified even as the object is returned and referenced by reverse. After reading the specification in more detail, I realized this behavior is intended. I was able to test this by using the Node environment, then performing

let a = [];
reverse = a.reverse();
a.push(1);

The result of this was that both a and reverse had [1] as the result. This is when I realized they are references and cannot be used to check and see if the array is truly empty. I then thought of comparing it by using

assert.sameValue(
reverse,[], `#1: a = []; a.reverse(); reverse.length === 0. Actual: ${reverse}`
);

This did not work because assert.sameValue() is checking the memory address for which the two variables were pointing to. The solution I came up with to guarantee that it is indeed what I expected was to use the length property of reverse to ensure the length is 0.

2) Checking values inside an array with length > 0

This problem is related to the previous one. My assumption was that I could check the contents of an array and make comparisons by passing in a separate array with the expected result.

a = [1, 2];
reverse = a.reverse();
assert.notSameValue(
a, reverse, `#2: a = [1, 2]; a.reverse() !== a. Actual: ${reverse}`
);

The result was the following

FAIL array-with-primitives.js (default)
#2: a = [‘a’, ‘b’]; a.reverse() !== a. Actual: 2,1 Expected SameValue(«2,1», «2,1») to be false

FAIL array-with-primitives.js (strict-mode)
#2: a = [‘a’, ‘b’]; a.reverse() !== a. Actual: 2,1 Expected SameValue(«2,1», «2,1») to be false

This failure indicated to me at the same time that when the specification mentioned Set() that the current values of the array were being modified. Meaning a's contents were being directly changed when a swap occurs. This makes sense since the specification takes this as an argument to be made into an object. I’ve also realized after that when the object is return, this is similar to C++’s return this; statement, in which the address to the pointer that points to the underlying array in memory is being returned.

This has been confirmed when I investigated into SameValue(). SameValue(x,y) in this scenario should return SameValueNonNumber(x,y). SameValueNonNumber(x,y) states that if x and y are the same Object value return true. Otherwise return false. This further confirms my theory to be true.

However, there’s a second question that came to me. At a quick glance, I thought to myself, “Why did I fail 2 tests when clearly only one has failed?” The reason is because there are two tests that are being executed by the harness. The first test is executed in default mode. the second test is executed in strict-mode. Strict mode is a subset of JavaScript that removes all the features that are the typical leading cause to bugs.

The solution I have to this memory problem was to test each individual element separately.

a = [‘a’, 1, undefined, null];
reverse = a. reverse();
assert.sameValue(
reverse[0], null, `#4: a = [‘a’, 1, undefined, null]; reverse = a.reverse(); reverse[0] !== null. Actual: ${reverse[0]}.`
);
assert.sameValue(
reverse[1], undefined, `#4: a = [‘a’, 1, undefined, null]; reverse = a.reverse(); reverse[1] !== undefined. Actual: ${reverse[1]}.`
);
assert.sameValue(
reverse[2], 1, `#4: a = [‘a’, 1, undefined, null]; reverse = a.reverse(); reverse[2] !== 1. Actual: ${reverse[2]}.`
);
assert.sameValue(
reverse[3], ‘a’, `#4: a = [‘a’, 1, undefined, null]; reverse = a.reverse(); reverse[3] !== ‘a’. Actual: ${reverse[3]}.`
);

This worked, but it was tedious and clumsy.

3) When a test fails, all tests thereafter do not get executed.

This was a behavior that I did not expect. Considering the harness is a tool to check for the browser’s level of compliance, failing one tests shows “2 tests failed”. However, what confused me was that even after inducing 4 failed tests, the result was 2 failed tests. This is how I realized testing stops the moment a test fails. I’m not sure if this is intended behavior as running the harness with the original tests showed that the harness continued testing even after failing some tests. Comparing to tests written by others, I do not see anything different when compared to mine other than the method of testing I use being more up to date with the latest conventions. I’ve also considered if it is due to synchronous testing, but ruled it out when other tests did not use promises. I am not sure if this is intended behavior or simply a bug.

Step 7: Improving my tests based on inspiration from TypedArray.prototype.reverse

One thing I’ve noticed after observing all the files under test/built-ins/TypedArray/prototype/reverse/reverts.js was the use of a method called compareArray(). While it looks intuitive at first glance, I was curious to find out its inner working since I felt my existing tests could benefit from this function.

Using the “T” shortcut on GitHub, I was able to locate a file named compareArray.js under the harness directory. The function was straight forward and returns a boolean if two arrays are equal in their values. This is exactly the function I need!

I rewrote my tests. This is what it looks like now

Conclusion

Overall, I’ve learned how to get involved in the standardization process of the web, and in particular ECMAScript. I now know how to setup the software necessary for standards testing and how to make use of GitHub’s README.md and CONTRIBUTE.md to contribute to open standards. I’ve also learned more about how JavaScript behaves by doing this exercise. The specification for Array.prototypes.reverse() seemed cryptic at first, but after re-reading the pseudo code a few times, it became clear what its intentions were. It is satisfying to see that all the test you’ve written are passed by Node.JS. I got a glimpse of how great developers build simple functions and unit testing code that is only one to two lines long. It was incredibly easy to understand and pleasant to work with.

Working on my first open source bug

As a first attempt in working on an open source bug, I’ve looked into the VS Code project where the color picker no longer appears when the user hovers over the color decoration. The first step in attempting to tackle a bug is to be able to reproduce the bug. Please keep in mind, all of these actions were done on my Mac.

How I reproduce the bug

1. Launch VS Code in developer mode.

Since I’ve already compiled the program from the previous post, all I had to do to run the software in developer mode was execute ./script/code.sh in the vscode repository directory.

2. Open command pallet using (cmd + p)

3. Typed in “user settings” inside the command pallet and hit enter.

4. Created a workbench color decoration option using “workbench.colorCustomizations” with an object as value containing {sideBar.background: "#ff0000"}

5. Hover my mouse over the red colored decorator and saw that the color picker did not appear.

One important note is that the issue suggests, by hovering the mouse on top of the text next to the color decorator, the color picker would appear. I’ve tested this to be true also.

Tools I used

The tools I ended up using in attempts to resolve the bug was Google Chrome Developer Tools within VS Code and GitHub Issues.

What I tried

The simplest yet direct method is to use Google Chrome Developer Tools to inspect the UI for changes. Since this issue touches upon issues relating to the UI of VS Code, I’ve set a DOM Manipulation breakpoint on the text next to the color decorator element by right clicking on the element and selecting break on “attribute modifications”. I used the DOM element selector tool to find the HTML element that is responsible for rendering the text next to the decorator. The DOM manipulation breakpoint was set at that location because it is an element that I can invoke the color picker from that is very close to the color decorator. However, this didn’t yield any results because when I hovered my mouse on the text and the selected a different color from the color picker, the breakpoint did not trigger to allow further investigation.

I then turned to closed GitHub issues to get a better idea of where the source file responsible for rendering the color decorator and the text maybe. I’ve found 2 closed issues: 1) Settings to hide color preview boxes. 2) Color picker alters RGB components when changing opacity. Both issues have a commonality of having commits made against src/vs/base/common/color.ts. I decided to look for this file to set breakpoints and run the code to see if my assumptions were correct. However, when I attempted to open the file, I was only able to find src/vs/base/common, but not color.ts.

Upon getting advice from Professor Humphrey, I was told that the code for the color decorator runs on a separate process from the main UI in the extension host process and that I’ll need to attach VS Code to that process to debug.

Can I fix it?

I believe I can fix it once I find color.ts and the methods associated with the color decorator and the text next to the color decorator, which activates the color picker. However, at this point, I will need further assistance before I can continue with the fix.

What about tests?

While I could not provide a fix at this point, color.ts does indeed have a test file associated with it. The file is color.test.js located at src/vs/base/test/common/. This file will prove useful once I’ve made changes to the code in attempts to fix the issue.

Obtaining VS code project and getting started in open source using a Mac

Installing VS Code as a user

The process was a breeze. I simply navigated to code.visualstudio.com, clicked the download button and everything else was history.

Extensions Installed

I’ve come from a background of having used the Atom editor as well as Sublime text 3, so I have a few plugins that I really like. In addition, there were a few I installed later on that is required to work on the developer version of VS Code. They are all great tools that help troubleshoot or extend a developers capabilities to do more.

• Google Chrome Developer Tools: Required for VSCode development
• JSHint: A JavaScript syntax checker before error messages hit the fan
• TSLint: A Typescript syntax checker before error messages hit the fan, used for VSCode development
• VIM: I love VIM editor shortcuts. It makes writing code super fast
• VS-Code icons: Just to spice up the look of the editor
• Prettify JSON: I do a lot of web programming, especially with web service so it is nice to have formatted JSON that’s human readable
• TODO Highlight: A tool that allows you to customize words to highlight in the editor to make notes as you code (i.e. FIXME, REVIEW, TODO, etc.)
• Code Spell Checker: I’m not very good with spelling, so this will save me the embarrassment

My Custom Settings

{
"git.autofetch": true,
"editor.tabSize": 2,
"editor.fontSize": 14,
"editor.fontFamily": "SFMono-Regular,Menlo, Monaco, 'Courier New', monospace",
"editor.wordWrap": "wordWrapColumn",
"editor.rulers": [80],
"files.exclude": {
"**/.git": true,
"**/.svn": true,
"**/.hg": true,
"**/CVS": true,
"**/.DS_Store": true
},
"window.zoomLevel": 1,
"cSpell.language": "en",
"workbench.iconTheme": "vscode-icons",
"todohighlight.isEnable": true,
"todohighlight.keywords": [
{
"text": "TODO:",
"color": "#3f3f3f",
"overviewRulerColor": "grey",
"isWholeLine": false
},{
"text": "NOTE:",
"color": "#e0e0e0",
"backgroundColor": "#0e8738",
"overviewRulerColor": "grey",
"isWholeLine": false
},
{
"text": "REVIEW:",
"color": "#e0e0e0",
"backgroundColor": "#6302d3",
"overviewRulerColor": "grey",
"isWholeLine": false
},
{
"text": "DEBUG:",
"color": "#e0e0e0",
"backgroundColor": "#0045f9",
"overviewRulerColor": "grey",
"isWholeLine": false
},
{
"text": "HACK:",
"color": "#000",
"overviewRulerColor": "grey",
"isWholeLine": false
}
],
"editor.minimap.enabled": false

Here are the things I’ve changed for the editor:
• Disable minimap: Since I’m on a 13″ MacBook Pro, screen real estate is very precious. The mini-map significantly reduces the available space to display code. I make up for this by using find (cmd + f) or using VIM plugin to scroll up and down via (ctrl + F and ctrl + B).

• todohighlight settings: These settings as mentioned above, allow me to customize words and the colors used to present them on the editor. It improves my workflow by allowing me to make notes that I can refer to as I am developing

• cSpell: This setting is for the spell checker explained above and specifies English and the language.

• editor.rulers: Specifies an array of numbers for which the editor should include a vertical ruler. This is helpful to see if your text is too long and is wrapping to the next line.

• editor.wordWrap: Specifies how the editor should perform wordWrap. In my case, I’m using soft wrap. This means if the text overflows past 80 characters, the editor will automatically wrap my text instead of me having to do it manually.

• tabsize: I like using only 2 spaces for tabs to conserve space and display more code.

• editor.fontFamily: This is very important to me because it is the very text I read every time I’m coding. I have it in the following order “SFMono-Regular,Menlo, Monaco, ‘Courier New”. The first font is the monospaced version of the new San Fransisco font that apple has released about 4 years ago. It is smooth, modern and really easy on the eyes. The others are also my favorites, but in descending order as they are more dated fonts.

Technologies used by VS Code

• Electron – Electron is a framework by GitHub that supports building native apps on desktop/laptop using web technologies. It uses Node JS as a back-end and Chromium to render the UI to the user.

• Typescript – A new language that is a super set of JavaScript created by Microsoft that transpiles down into JavaScript. It’s special feature includes compile time type checking, support for interface, enum, generics, tuple, and namespaces.

• Mocha – A Node JS framework used to perform unit testing to ensure code runs as expected as multiple developers collaborate together and the application becomes more complex.

• Sinon – A spy/mock testing tool for Node JS. This tool helps with unit testing by mimicking data and results (mocking) or inspecting data as it passes through functions (spies) to ensure the application is working as expected. It is used in conjunction with unit testing frameworks such as Mocha.

• Yarn – A package installer created by Facebook to install Node JS packages. It is lightweight, uses checksum for package integrity, allows offline reinstalling, and automatic retry on download failure. It also supports a .lock file to ensure packages installed on one system will be installed exactly the same way on another.

• ESLint / TSLint – Both are syntax checkers for JavaScript and Typescript respectively. As the developer is working on his/her code, the linters will display red squares/dots on the editor to note errors in the code before the code gets compiled or executed.

• Node JS – A JavaScript environment that is built on Google’s V8 JavaScript engine. It’s purpose is to provide a run-time environment for JavaScript applications that is lightweight and super fast. The primary use of Node JS is for web development because of its event-driven and non-blocking I/O capabilities, which makes all requests asynchronous, allowing web apps and services to be more responsive. It also features Node Package Manager, a popular package manager to install modules to extend its capabilities.

Getting Started

As a lab, my professor David Humphrey got me started on working on Microsoft’s VS Code. It is an amazing text editor that combines the best of Sublime Text editor with Microsoft’s intellisense code completion system. Furthermore, it has a built-in debugger, terminal and of course, a large list of plugins to choose from.

Here’s the process I went through:

1) Fork the repo from Microsoft’s VS Code github.

2) Clone the repo to my local machine by running git clone https://github.com/AlexWang-16/vscode.git.

While I’m using the link to my repo, you can make this work for your own repo by switching out the link with your own.

3) Add Microsoft’s VSCode repo as remote upstream. This allows getting the latest updates from Microsoft while working on a bug fix or new feature. You will notice I’m using SSH version of the link here. This is to simplify interacting with GitHub without prompting for my login each time. You can set up SSH keys for machines you frequently use here.

git remote add upstream git@github.com:Microsoft/vscode.git

4) Install all of its required dependencies based on how to contribute wiki page inside their repoistory. Since I’m on a Mac machine, I’ve installed Python 2.7.14 via Homebrew, XCode LibTools, Node via Node Version Manager (NVM), and yarn package manager.

I did not install git because git was shipped as a standard development tool from Apple and while it is not the latest version, it is sufficient for my needs.

It is important to note that I used NVM during the installation of Node because Microsoft has specified that only versions >= 8.9.1 and < 9.0.0 are supported.

To install a specific version of node using NVM execute: nvm install 8.9.1

5) Execute yarn via terminal in the repo’s directory to build the developer version of VS Code. At this point, I ran into some issues, which I will detail later.

6) I then executed ./scripts/code.sh

However, at this point, I’ve ran into more problems.

Problems, problems, more problems

The first problem I ran into was the compiler complaining about a missing plugin NSFW. This plugin is associated with Electron, a framework that allows users to develop native applications on desktops/laptops using web technologies (Node JS back-end and Chromium front-end).

I was able to avoid this initially by going directly into the /build directory and running yarn, but the problem eventually would come back to haunt me.

The second problem that occurred was when I ran ./scripts/code.sh to launch the editor. The error message was:

* Activating extension `vscode.merge-conflict` failed: Cannot find module ‘vscode-nls’.

* Activating extension `vscode.git` failed: Cannot find module ‘vscode-nls’.

* Activating extension `vscode.emmet` failed: Cannot find module ‘@emmetio/html-matcher’.

The solution I’ve found was to execute npm run postinstall inside build/

NSFW Won’t install

The NSFW issue came back while I was attempting to run tests to verify that the code base did not have issues before I begin working on the repository. When I executed ./scripts/test.sh, there was a large amount of errors being returned. This is how it looks like:

[11894:0212/151338.307494:INFO:CONSOLE(270)] “[object Object]”, source: /Users/awang05/dev/vscode/test/electron/renderer.js (270)

* Executed ./scripts/test.sh –debug as suggested by David Humphrey to get more descriptive error message

Error: Cannot find module ‘../../build/Release/nsfw.node’ at Module._resolveFilename (module.js:470:15) at Function.Module._resolveFilename (/Users/awang05/dev/vscode/.build/electron/Code – OSS.app/Contents/Resources/electron.asar/common/reset-search-paths.js:35:12) at Function.Module._load (module.js:418:25) at Module.require (module.js:498:17) at require (internal/module.js:20:19) at Object. (/Users/awang05/dev/vscode/node_modules/nsfw/lib/src/index.js:3:16) at Object. (/Users/awang05/dev/vscode/node_modules/nsfw/lib/src/index.js:130:3) at Module._compile (module.js:571:32) at Object.Module._extensions.js (module.js:580:10) at Module.load (module.js:488:32)

With the help of my professor, I executed ./scripts/test.sh --debug to gain more insight into the problem.

Since we’ve noticed NSFW was missing, we tried installing NSFW via yarn yarn add nsfw. However, yarn warns that XCode Alcatraz package manager is missing. So the deep dive into the issue continues.

2018-02-12 15:39:59.752 xcodebuild[13996:3090356] [MT] PluginLoading: Required plug-in compatibility UUID B395D63E-9166-4CD6-9287-6889D507AD6A for plug-in at path ‘~/Library/Application Support/Developer/Shared/Xcode/Plug-ins/Alcatraz.xcplugin’ not present in DVTPlugInCompatibilityUUIDs

2018-02-12 15:39:59.753 xcodebuild[13996:3090356] [MT] PluginLoading: Required plug-in compatibility UUID B395D63E-9166-4CD6-9287-6889D507AD6A for plug-in at path ‘~/Library/Application Support/Developer/Shared/Xcode/Plug-ins/AdjustFontSize.xcplugin’ not present in DVTPlugInCompatibilityUUIDs

After having found how to add UUIDs for the required plugins, I executed defaults write MyPlugin.xcplugin/Contents/Info DVTPlugInCompatibilityUUIDs -array-add B395D63E-9166-4CD6-9287-6889D507AD6A. However, this did not work.

I then found an alternative from here and executed

defaults write ~/Library/Application\ Support/Developer/Shared/Xcode/Plug-ins/Alcatraz.xcplugin/Contents/Info DVTPlugInCompatibilityUUIDs -array-add $(defaults read /Applications/Xcode.app/Contents/Info DVTPlugInCompatibilityUUID)

defaults write ~/Library/Application\ Support/Developer/Shared/Xcode/Plug-ins/AdjustFontSize.xcplugin/Contents/Info DVTPlugInCompatibilityUUIDs -array-add $(defaults read /Applications/Xcode.app/Contents/Info DVTPlugInCompatibilityUUID)

The result from executing these two commands were

/Users/awang05/Library/Application Support/Developer/Shared/Xcode/Plug-ins/AdjustFontSize.xcplugin/Contents/MacOS/AdjustFontSize: code signing blocked mmap() of ‘/Users/awang05/Library/Application Support/Developer/Shared/Xcode/Plug-ins/AdjustFontSize.xcplugin/Contents/MacOS/…

At this point, I got more assistance from my professor and realized that I had Macports and Homebrew installed. To eliminate potential conflicts between Macports and Homebrew, I uninstalled Macports.

Then I performed the following in the terminal.
1. brew install python 2
2. brew install libtool

We want to install homebrew’s version of libtool to bypass Xcode’s libtool since it tends to be more up to date and supports more features.

However, Homebrew notes that I already have this tool installed (from XCode) and that I only need to link to it to make it work. Naturally, I executed sudo brew link libtool. However, for security reasons, Homebrew now refuses to perform any action via sudo since it provides the execute script with root permissions.

To grant permission to the local system to perform linking, I tried sudo chown -R $USER:admin /usr/local/share/libtool/. However, this lead to another issue:

Error: Could not symlink include/libltdl/lt_system.h
Target /usr/local/include/libltdl/lt_system.h
already exists. You may want to remove it:
rm ‘/usr/local/include/libltdl/lt_system.h’

To preserve these files, I’ve attempted renaming them by adding a “.old” extension to the end of the files. However, the same error message would appear with a different file name each time. At this point, for the sake of sanity, I used the following command:

brew link --overwrite libtool.

Crossing my fingers, I ran yarn inside vscode directory hoping it would work. Yet, the same error occurred. However, we’ve noticed this time, it complained that a --static flag not being supported:

libtool: unrecognized option `-static’

Further investigation has led to the conclusion that my path was pointing to the version of libtool that does not support the static flag (very likely the original libtools shipped with XCode). To solve this issue, I renamed /usr/local/bin/libtool to /usr/local/bin/libtool.old (to preserve the old version).

The path now points to homebrew’s libtool: /usr/bin/libtool.

Now VSCode compiles and all tests work.

First Pull Request to an Open Source Project

This post is a continuation from my previous post on the google phone number library web service project. I want to begin by stating that while I thought contributing to another person’s repo of the same project would be easy as I’ve done a bulk of the work myself, this was not the case.

The Difficulties

The biggest issue was trying to debug why unit testing wasn’t working. Initially, I thought it was the unit testing framework because I could use Postman, a HTTP client application to test the web service, to create desirable results. However, every time I ran the unit test, I would get an error that says

res.json is not a function

This didn’t make sense. To elaborate on the issue, “res” is the variable for the HTTP response received by mocha after sending a file containing phone numbers to the web service for parsing. I tried the following:

1. Console.log(res) to see if the variable contains data and it does!

2. Using VSCode’s debugger to step through the test code execution as suggested by Sean Prashad.

The problem I’ve encountered shortly was that the code would jump into Express JS’s library. I began hopping through code that I didn’t understand and I did not manage to come out of it within a reasonable amount of mouse clicks.

The solution

I thought, if I could start the server just fine, then I should trace the steps in the code and compare it to my test code setup to see what was different to get insight into what was wrong.

As it turns out, the test script had incorrectly required the route file instead of the web service app itself. It immediately became clear why it wasn’t working. Mocha was not using the web service app for testing, but a route of the service app.

Contributions

In total, I’ve a few contributions to the repository.

• Fixed unit tests to ensure they all work
• Added the ability for the web service to decode Base64 formatted text files
• Added the ability for the web service to handle invalid GET requests (i.e. if the user provides no phone numbers to /api/phonenumbers/parse/text/ , the web service returns an empty array as expected)
• Added tests for new functionality as listed above
• Added Travis CI support for the repo

What I learned

During the entire process, I had to research a number of things. The first thing was Base64 conversion. Up to this point, I had no idea that files uploaded to servers should be Base64 encoded. In addition, I do not know how to encode or decode the text itself. Luckily, JavaScript already provides that functionality.

Troubleshooting process

The second thing I learned about was the troubleshooting process. After this incident, I realized the way to debug faster is to trace the code backwards, from the point of failure and to consider all dependencies that the code depends on. While my methods may change in the future as I gain more experience, this is all I know thus far to be best practice.

Debugging with Mocha

Third, it was interesting to learn about debugging using the VSCode editor. If you’ve used Visual Studio in the past, you will feel very at home with VSCode. Keyboard bindings such as F5 to start with debugging and Ctrl + F5 to start without debugging were all the same. The gutter areas between the line number and the edge of the editor are still the areas you work click to mark break points to name a few. There are lots more similarities. Here’s what I did to configure and start my debug session with mocha:

1. File > Open

2. Find and select the folder for your application

3. Click on the debug icon Screen Shot 2018-02-11 at 12.53.17 AM on the left menu bar, then select the settings button Screen Shot 2018-02-11 at 12.53.44 AMnext to the play button.

4. With your mouse, click inside the array of “configurations” property and click “Add configuration” button.

5. Select “Node.js Mocha test”

Screen Shot 2018-02-11 at 12.57.38 AM

6. Navigate to the part of my code where the issue is happening and set a break point by clicking on the gutter area of the editor between the left edge and the line numbers. A red dot should appear.

Screen Shot 2018-02-11 at 1.04.16 AM

7. Click on the play button to begin debugging with mocha.

Screen Shot 2018-02-11 at 1.05.22 AM

The debugger will then pause at those breakpoints and you will be able to expand the “variables”, “watch”, and “call stack” of your code at any given point on the left panel.

Travis CI

The last thing I’ve learned is how to use Travis CI and how amazing this new tool really is (once you understand how to set it up). While I thought it would be super challenging, it did not take as long as I had anticipated. The fact that this tool can automatically clone your repository, install the environment and dependencies based on your .travis.yml file, build your code and even run tests every time a push or pull request (or both) is done will make working projects in teams or alone much easier. It allows higher quality applications to the built by testing and building the latest version of the code frequently rather than waiting for one major release to be committed by a single developer all at once and testing after, which then results in long debugging and merging sessions.

I’m glad I’ve done this assignment. I feel much more prepared. The fact that I had to step through the process of forking a repository, creating a branch to work on a fix, then add, commit, and push those updates, and finally create a pull request has made me much more comfortable at contributing to open source software.

I look forward to contributing to a major project soon. In fact, Microsoft VSCode is on the list next!

Until next time.

Google phone library web service project

GitHub Repository: webservice-phonelibrary

The first submission for my open source class assignment has been made. The project works, but is far from complete. Looking back at my experience, I’ve learned quite a few things with respect to modules in Node.JS. For example, I’ve never used multer before as I did not need to process file uploads. From this experience, I’ve learned to search and read the API for another github project. Using multer, I was able to read from a file and delete a file (after finishing processing the file).

Since findNumbers() method was not defined from the google library and I was limited in time, I’ve decided to write a quick and dirty regular expression to handle North American phone numbers. This took some time as I needed to refresh my memory from the last time I’ve used regular expressions. On a side note, it is interesting to note that Express JS uses routes regular expression to detect and handle routing.

Some issues I ran into while creating this web service were errors when calling the code in the library. For example, when calling isPossibleNumber(), the error message I got was

a.getNationalNumber() is not a function

It was frustrating to find that there were no other methods I could call that worked, which allowed me to quickly verify if a number is valid or not.

With regards to unit testing, I’ve been using SuperTest for a while, but now I’ve learned how to perform tests with file uploads.

Furthermore, I’ve had for the longest time, a problem with testing the web service was I have two test files, one file tests the root endpoint, the second test file is responsible for testing the parse endpoints. When I execute the test with jest, the test files are ran concurrently using my app.js file. Having app.listen() inside app.js will result in collisions when jest attempts to start two test instances concurrently.

The solution as suggested by Professor Humphrey was to separate the code for starting the server with the app itself. I’ve created server.js which includes the logic to start the server, in app.js, I export the app variable to allow server to take over and start the server instance.

Once I’ve completed coding, I went to put my efforts into crafting my README.md file. I’ve also learned to create new issues on Github to allow others to contribute as well as allow issue tracking for a project.

This has been an interesting experience. I look forward to contributing to other’s projects and fixing their bugs.