Exploring the Google Chrome debugger protocol by trying to talk to ClearScript running in V8.

An enterprise app I develop & maintain makes use of ClearScript as an extensibility point: end-users (semi-technical) write financial algorithms in JavaScript and those algorithms are evaluated by the system during batch processing (certain key C# types are exposed that the algorithms can make use of). ClearScript is set up to run the JavaScript against the Chrome “V8” JavaScript engine (the same as NodeJS and, well, Chrome).

Some of these algorithms are quite complex and time consuming to write - the question arose “Can we provide a way for end-users to debug the algorithms”?

As a minimum-viable-product I decided users would need the ability to

• Set breakpoints
• Inspect the value of variables
• Step over/into code
• Modify the value of variables in-place (nice-to-have)

As noted my StackOverflow question, whilst I’m (as a developer) able to connect to the V8 engine w/ Chrome DevTools, that’s not ideal for end-users for a couple of reasons:

• Tricky to set up / not integrated with the rest of the web app
• Probably won’t play nicely with corporate firewall (every simultaneous instance of the debugger will have to listen on a unique port; That’s another port we’d need to let through the firewall).

I was therefore wondering - is there a .NET interface to the Chrome V8 debugger? I didn’t find one so set out about building one.

SIDE NOTE: After building mine, I found chrome-dev-tools-runtime - it looks pretty similar to what I implemented (but more/less elegant in places). Note that as well as a concrete implementation, the same author (BaristaLabs) offers a generator for creating DTOs for the various messages/events for the protocol (the protocol is constantly evolving as Chrome evolves). That should be less of an issue when targetting ClearScript though.

Even after reading what I could find in terms of official documtation here & here I still didn’t really understand how the protocol works. With a bit of Fiddler + Wireshark and general prodding I managed to talk to it. Here’s what I found:

• It’s primarily a websocket based protocol (asynchronous JSON messages). That said, the initial setup of the session needs to be done through HTTP (I’m not referring to the websocket protocol escalation).
• Client code can interact with the debugger by sending commands & (asyncronously) waiting for a response (commands have an id property which is used a sequence number to correlate requests and responses).
• The browser/debugger also publishes events to the same websocket - that slightly complicates processing of messages (since any given message could either be a response to a command or an event)

In the case of ClearScript hosted V8, the initial setup was as simple as making a GET to http://{hostname}:{port}/json/list - that returns a list of open “tabs”. In a ClearScript scenario (as opposed to real Chrome) I think we’ll only ever have one tab. The tab DTO has an id property - we need the id to setup the WebSocket:

await _socket.ConnectAsync(new Uri($"ws://{_hostname}:{_port}/{tabInfo.Id}"), cancellationToken);

Issuing a command is as simple as serializing the command to JSON and sending it to the websocket. The websocket should be read periodically for command responses & events (also JSON).

Other miscellaneous information:

• When ClearScript evaluates a script, a debugger statement won’t do anything unless there’s a debugger attached (just like real Chrome needs DevTools open). Use the flags EnableDebugging and AwaitDebuggerAndPauseOnStart

V8ScriptEngineFlags flags = V8ScriptEngineFlags.EnableDebugging |
                              V8ScriptEngineFlags.AwaitDebuggerAndPauseOnStart;
new V8ScriptEngine(flags);

• A client connecting to the debugger (via websocket) is not enough to satisfy AwaitDebuggerAndPauseOnStart - the client must issue the commands Debugger.enable & Runtime.runIfWaitingForDebugger - after that point, even if the client disconnects the script will continue to completion.
• There is an experimental feature in Chrome Devtools whereby you can get DevTools to show the debugger protocol commands it is executing (see screenshot below). Extremely handy to know about!

Some useful commands:

• Runtime.enable
• Debugger.enable
• Runtime.runIfWaitingForDebugger
• Debugger.getScriptSource
• Debugger.resume
• Debugger.evaluateOnCallFrame
• Runtime.getProperties

I’ve built a Communicator class to abstract the V8 debugger interactions. The interface looks as follows:

public interface ICommunicator
{
    Task Connect(CancellationToken cancellationToken);
    Task<string> SendCommand(string method, object parameters = null);
    Task<TEvent> WaitForEventAsync<TEvent>(CancellationToken token) where TEvent : IV8EventParameters;
}

Consuming could would instantiate the communicator (in a using block), Connect to it, invoke SendCommand several times, possibly wait on the DebuggerPaused / ScriptParsed events and finally Disconnect.

The implementation of Communicator makes use of MessagePump (a task running on a separate thread to poll & read messages (both events and responses to commands) from the websocket. The MessagePump raises events which the Communicator handles.

Things get a little bit fruity in the Communicator class - it creates three channels (one for each of the event types + another for command responses) and adds event handlers that write to the appropriate channel. The implementation of WaitForDebuggerPausedEventAsync and WaitForScriptParsedEventAsyncis trivial - we just read asyncronously from the appropriate channel. Obviously, this approach wouldn’t scale if we were interested in multiple types of events. Calling could would probably like a generic WaitForEventAsync<TEvent> method anyway. I’m not sure how you’d achieve that.

UPDATE: 3rd September 2019: I’ve refactored things to remove the channels from the Communicator class - instead, the code now maintains a dictionary of buffered events as well as a dictionary of task completion sources. When an event is fired or awaited we will pair of the first matching task completion source with the (possibly buffered) event.

A sample solution (showing a ClearScript target script being debugged by another process) is up on Github if you’re interested.

Conclusion

Whilst I was able to achieve the goals I set out to (set breakpoint, step over/into, examine/modify variables) there’s clearly a lot of moving parts - integrating it into the production app would require careful management of threads to ensure things don’t get stuck / resources exhausted.

Hope you enjoyed the post!