Issue#5446, Pull Request#6058.
This is a pull request blog post. Please see my blog post on Mozilla devtools-html/debugger.html for more information about the project.
What’s the bug about?
Project maintainer @JasonLaster would like to add a mochitest that tests clicking the pause button on the debugger results in the debugger pausing on the next execution.
Chosen growth
The goal of this release was to practice writing unit tests for an opensource project. Unit testing has been a primary concern for me since I’ve had exposure to it about a year ago. I value quality software and understand that it is significant in both private companies and open source communities. Therefore, it is something that I would like to make second nature to me when I program. This was the perfect opportunity.
Adding the test
The first step to fixing a bug for any opensource software is to fork the repository. Then, clone the forked repository onto the local machine and create a new branch:
git checkout -b issue-5446
The second part is learning more about mochitests. In short, mochitests are unit tests that use the MochiKit platform (an opensource project that Mozilla uses to test the Firefox browser). Using mochitests, developers can programmatically simulate actions that users will perform on the Firefox debugger.
Following the instructions on the mochitests documentation, I’ve setup mercurial (a git like version control system) and autoconf213, a script that produces shell scripts that configures source code packages.
Next up is downloading a copy of Firefox into the repository and configuring links between debugger.html/ directory and Firefox’s test directory. Luckily this can all be done using a simple command ./bin/prepare-mochitests-dev
Once the apparatus was setup, it was time to begin researching about how to write mochitests. @JasonLaster has given me some hints by pointing me to a specific test to look for. He suggested that I look at browser_dbg-breaking.js inside src/test/mochitest directory.
@JasonLaster has outlined the steps the test needs to perform in the issue. The steps were as follows:
1. Load a page
2. click the pause button on the debugger
3. Eval a function
4. Assert that we are still paused
The first problem I encountered as that the pause button did not exist.
This was resolved by changing the debugger’s settings:
1. In the debugger’s console execute: dbg.features
2. set “remove-command-bar-options” to false
The second part was understanding how to write the tests. The most difficult parts being clicking the pause button on the debugger and evaluating a function. After getting some clarification, I proceeded to experiment with the API to find out the relevant functions to use. The API for the tests were located in src/test/mochitest/head.js.
Based on intuition, I first attempted with pressKey(dbg, "pauseKey"). This simulated pressing on the pause key on the keyboard. However, this did not work. There was also no obvious hints I can grasp from the API at that point, so I turned to using git grep to help me find the use of the word pause in other tests under src/test directory to get more hints. Professor Humphrey has taught me to use git grep -C numberOfLines searchPhrase path as a means of being able to see the number of lines surrounding the search phrase.
While experimenting with the API, something really odd had occurred. The symbolic linking had suddenly been undone, which lead to the harness completely break. Unfortunately, this only occurred one time and so I was unable to reproduce the problem or find the root cause of it.
I tried deleting the entire directory, re-cloning the repository and initiating the configuration process. However, during the configuration process, there were error messages showing that I did not understand. After googling around, I was unable to find answers as the problem is specific to this project. I reached out on slack to get assistance.
@JasonLaster mentioned in the chat to debug issues with the harness, go into the firefox/ directory inside the project folder and run ./mach mochitest --headless devtools/client/debugger/new.
After following those instructions, I was able to diagnose that my computer was missing the Rust compiler. To fix that issue, I ran ./mach boostrap to try to get the compiler installed, but ran into another error. I was suggested to run ./mach configre first.
After bring presented with four flavors of firefox to select from, I was instructed to build using Firefox for Desktop Artifact Mode. From there on, I re-ran ./bin/prepare-mochitests-dev and after 5 minutes later, I was back in business.
Not wanting to waste anymore time, I turned to looking at test files individually based on the relevance of their names to gain additional insight. I saw some UI tests using the function clickElement(). I looked inside head.js to look for the definition of clickElement. I’ve found that it takes the debugger instance as the first argument, but then a selector as the second argument. The list of selectors were defined as a dictionary. Scanning the list of selectors, I did not see one for pause even though one for resume existed. The value for the resume was “.resume.active”. I began taking guesses that this may have to do with the user interface.
I initiated an instance of the debugger to confirm my hypothesis. Indeed, after using the inspector on the DOM, I was able to see that “resume” and “active” were both CSS class of the resume button. I then inspected the pause button to see what that said and it turns out to be “.pause.active”.
Executing the test with await waitForTime(15000) gave me a 15 second window to play around with the pause button under test mode to see outputs that were being logged to the console. In particular, when I clicked on the pause button, “1 BREAK_ON_NEXT” appeared. This was a hint for a command that I should listen for to ensure that the pause button was clicked!
I went back to head.js and added the property “pause” and value “.pause.active” to the selectors dictionary.Then, inside my test, I added await waitForDispatch(dbg, "BREAK_ON_NEXT").
However, once I’ve applied this, I began getting errors about cyclical references. It took a while for me to understand what was happening. I got additional help from @bomsy who confirmed that I was on the right track with defining a selector for paused to simulate clicking the pause button.
After much digging, I realized the problem came from my improper use of invokeInTab(). I had passed the debugger into the function which lead to self-invocation to infinitude.
The next step was to figure out how to eval a function. According to @JasonLaster, I should be able to use invokeInTab() to simulate executing a function from console. However, this did not seem to work. The test repeatedly timed out as it waited for the next action to execute after the pause button was clicked.
Since I was unable to trigger this myself via the console, my temporary work around is to execute the function as an expression in the watch window, which is considered an execution to trigger pausing the debugger.
I was able to find an example of this under src/test/mochitest/browser_debugger-expressions.js.
In addition, I’ve added an extra line of code to tell the debugger to wait for the debugger to pause to give it time for the pause effect to kick in; All of the test code is executed asynchronously and therefore required await if it needs to wait for the command to resolve. Otherwise, race conditions would occur and the test may fail due to code execution not yet completed.
I have since submitted a pull request and currently waiting for @JasonLaster to review the pull request. Additional updates will be made to this post at a later date once the review has been completed.
What I learned
While writing a test seemed very simple, a lot of the time was spent setting up the test harness and understanding the available API. While the Firefox Debugger team had documentation on mochitests, there were some areas that were unclear. In particular, there were no instructions on how to troubleshoot issues when Firefox does not compile. Inside head.js, some functions were not documented and required inference and guesses on what those functions do.
I felt that I’ve done better this time around having reached out to the community for help a lot sooner than my first pull request. I’ve practiced using git grep to help me find specific keywords that I am interested in while searching for example usages of the test harness API that I was learning. Furthermore, I’ve expanded this concept to looking at test files that other developers have written to understand the usage of other commands, which I may not have been aware of initially.
What surprised me the most is how the test harness can suddenly break and the importance of documentation and community to help opensource developers get through the hurdles of having the environment steady enough for them to tackle the real issue at hand. I saw first handed how big opensource projects can get and that work is never complete, just more work to be done.
opensource adventures 