adrift on a cosmic ocean

Writings on various topics (mostly technical) from Oliver Hookins and Angela Collins. We currently reside in Sydney after almost a decade in Berlin, have three kids, and have far too little time to really justify having a blog.

Reflecting on TypeScript, Lambdas, and of course testing

Posted by Oliver on the 27th of September, 2018 in category Tech
Tagged with: testingunit-testingjavascriptTypescriptLambda

Yet another long-overdue post. The last couple of weeks (or perhaps sprints) my team has dived into Typescript for a new project. I like it, really like it. Maybe that's just because for years I've feared Javascript, haven't understood enough about how to unit testing, and found myself writing terrible code and just running it to find the bugs. This is not a very good development methodology.

Typescript and IDEs

This time around I'm using a great IDE (Visual Studio Code, of course), have type packages for just about everything we are working with, and am writing unit tests. I have persisted for many years just using Vim (in the recent history, mostly for writing Go), and because my job has frequently been quite varied and I'm often too lazy to install plugins or custom editor scripts, it didn't have anything really helping me with my coding. I think the only times in the last 8 or 9 years I've genuinely wanted, needed and used an IDE has been for writing Flash ActionScript 3.0 and Java (IntelliJ IDEA in both cases). That's not counting the dabbling in Unity/UnrealEngine where I've used MonoDevelop or Visual Studio.

But I digress. VSCode is working great, it supports many languages and is vastly improving my coding experience - I think I'll keep using it. So, my brief recipe for getting into Typescript is (and I wrote up this short guide for my team recently as well):

  • Install and use VSCode
  • Plugins: EditorConfig, npm intellisense, TSLint
  • Read some of the Typescript docs but not too much. I also read through quite a bit of Basarat Ali Syed's excellent Typescript Book which also goes a bit into the Javascript aspects and how to ease yourself in gently.
  • Rename your files from .js to .ts, open them in the editor and start fixing problems!
  • Start adding types and interfaces to things.

Before diving in, I'd had a look at some Typescript codebases that some of the teams around us are either creating from scratch, or migrating, and it seemed a bit daunting. My eyes glazed over at all the extra stuff and I went away for a little bit. Starting from scratch yourself, or even porting some Javascript ends up not being that much effort. We already found a number of bugs with some simple code, and after you have gotten used to interfaces (which are not too different to those from Go, I find) it feels natural adding them to code.

So, in summary, I recommend it!

Lambdas and Testing

This leads me to the use case - writing some code for Lambda@Edge, which can only use Javascript at the moment. When AWS came out with regular Lambdas, I created a proof of concept Lambda in Java (since I wanted the benefit of types) and found the testing story to be somewhat lacking. Admittedly this is much easier with Javascript since types are flexible, but with Typescript you add some of that brittleness back in.

A concrete example is around the interface to the handler function itself. By default you have something like (event, context, callback) as your interface to implement in your handler function. Event and context here can be fairly large data types that contain many properties - the best mitigation being to create as small a handler function as possible, and immediately extracting just the properties you need to pass to other functions with far smaller interfaces - this makes them much easier to test.

So far we have been fairly successful with this. One part of our code sadly has to deal with external network dependencies, seemingly an inevitability of life. Unit testing this of course requires mocking or stubbing out these dependencies and testing the code around it. During this process I learned some more about Javascript testing and interesting effects of asynchrony. Consider the following theoretical code:

let clock = sinon.useFakeTimers()
let cname = ''
let expectedAnswer = ''

let answer = await CachingResolver.ResolveCname(cname)


This will probably raise your eyebrows a bit, so it deserves some explanation:

  • We'd like to cache DNS resolution records locally for a short period of time to avoid lots of unnecessary network overheads (and potential failures) from slowing down the Lambda execution.
  • Network failures are possible though, so the resolver should timeout quickly if it can't reach the upstream.
  • In order to mock out a real DNS server, we need to mimic the same interface which also involves using setTimeout and promises.
  • But, we don't want to have to wait for real time to pass for our tests to complete. Especially in the case of simulating timeouts - we can introduce test timeouts unintentionally (mocha by default will timeout tests after two seconds).

The above code won't actually work, I realised. The caching resolver is calling a fake resolver which is using setTimeout - but time is standing still due to faking the timers. Now there is a conundrum since we can't call await on the DNS resolution due to it halting until the clock cranks forward. But, we can't crank the clock forward before asking the resolver for an answer, because then it won't know we want one!

The solution is to go back to promise chaining:

  let promise = CachingResolver.ResolveCname(cname)
    .then((answer: string[]) => {
  return promise

Now we don't wait on the resolution call, and just return the unsettled promise, and chain the expectations on the resolved promise result. Execution flows immediately on to the next line where we crank the clock forward (causing the resolution to start and expectations to be tested - however, to ensure that the test stanza doesn't exit early, we have to return the promise to it. The test harness ensures that the promise is resolved before wrapping up execution of the test case.

Maybe this is obvious to you seasoned Javascript professionals, but to me who is relatively new to Sinon, Mocha/Chai and generally testing in Javascript, it was a very interesting dive into the difficulties of timing in Javascript and how te manage asynchrony. My overall feeling about Javascript is still that there is a lot of boilerplate and effort in general just used to manage the concurrency mechanisms, but at least in this case with Edge Lambdas, for the moment we don't have a choice!

What does this testing problem have to do with Lambdas though? We still don't entirely understand how the event loop and tasks are managed exactly inside that runtime environment. Theoretically execution of our code finishes once we call the callback function (or with Node 8, once the promise is resolved or rejected). But it is possible to schedule later execution with setTimeout - are these timers still running? Are we billed for the whole time or just when tasks are executing? Interesting questions, and some I've never had to deal with before.

On the other hand, it's always nice to learn something new, and especially if it has utility and potential to bring business value. There are still some operational issues to figure out with Lambdas, especially in the deployment and monitoring side, but I'm confident the tooling will continue to improve as more people use it.

© 2010-2022 Oliver Hookins and Angela Collins