mykter.com

What does the AWS security certification mean?

Sat, 04 May 2019 00:00:00 +0100

I recently passed the strangely titled “AWS Certified Security - Specialty” exam, woo 🎉. I guess that makes me “AWS Security - Specialty” certified? Quite a mouthful.

This is what AWS say this means (numbering + split are my additions):

Abilities Validated by the Certification:

An understanding of specialized data classifications and AWS data protection mechanisms

An understanding of data encryption methods and AWS mechanisms to implement them

An understanding of secure Internet protocols and AWS mechanisms to implement them

A working knowledge of AWS security services and features of services to provide a secure production environment

Competency gained from two or more years of production deployment experience using AWS security services and features

Ability to make tradeoff decisions with regard to cost, security, and deployment complexity given a set of application requirements

An understanding of security operations and risk

First, note that there isn’t an entry along these lines: “Ability to make effective use of AWS security features to secure a production deployment”. I think that is what many people would expect this certification to validate. Number 6 comes close, but isn’t quite as general.

OK, so the certification is perhaps not quite targeted to what would make it most useful, but how well does it validate the abilities it does claim?

I’ve split the list into two halves – the top set deal primarily with knowledge of AWS services, the bottom set are more about experience and capabilities.

Knowledge of AWS security features

I think the certification does a good job at 1-4. It’s a multiple choice exam that covers a fairly broad swathe of AWS products. Most of the questions in my exam had at least one distracter that was sufficiently plausible to put you on at best 50:50 odds for the question if you didn’t know your AWS security features pretty well (usually the distracters either referenced a real AWS feature or product that doesn’t quite meet the requirements, or a perfect sounding feature that doesn’t actually exist).

This comes with an interesting caveat due to the nature of multiple choice questions: the right answer is always staring you in the face. The exam doesn’t test the ability of candidates to come up with possible solutions to a given problem, only to select a good solution from a set of options. Does this matter in practice? I suspect it does – the majority of your AWS security challenges are not going to be of the form “which of these should I do?”, they’re either going to be “how do I do X”, or even more subtle: knowing that there is something you should be doing in a particular area in the first place.

Applied security engineering in an AWS environment

The next set of claimed abilities are a lot harder to validate, and I don’t think the certification does a particularly good job here.

Number 5: 2+ years of production deployment experience. Firstly this is an extremely hard thing to define what it even means in terms of abilities, and secondly from my sample size of one it’s demonstrably false. I have at most 0.5 years production deployment experience with AWS security services – I started working seriously with AWS about 6 months prior to taking the exam. Perhaps my background in security and fairly intense learning over the past few months is roughly equal to a ‘typical’ 2 years of production experience? If you could reliably deduce this kind of information from a multiple-choice quiz, the technical part of hiring would be a solved problem.

Number 6: ability to trade off cost/security/complexity. This is really hard to test in a multiple choice exam, and certainly the questions I got in my exam (they’re drawn from a big pool of questions) didn’t assess this in any meaningful way. As soon as you’re trading off various valid answers that only vary in domains like cost/complexity/security, either (a) the question has to be quite contrived and very leading, or (b) you need way more context than they include about a scenario to be able to determine the ‘right’ answer, and even then you might have in mind some reasonable assumptions that make an alternative answer valid. Multiple choice questions need a clear best answer, and don’t allow for “it depends”.

Number 7: security operations and risk. I’m least confident about my assessment of this one, as it’s the area I’m most comfortable with – I don’t remember much about the questions I got in this domain. Again though, I think most of the questions were pretty basic or fairly leading.

In brief

To sum up, I think the cert does a good job of testing your knowledge of AWS security services and features, and not such a good job of testing that you can use that knowledge effectively.

OK that’s it! If you too have less than 2 years production experience of AWS security but want to pass the exam, check out the notes I made as I was studying. It’s a big list of all the AWS-specific security topics that cover points 1-4:

Just passed the AWS security specialty certification exam 🎉

Here are the notes I made as I went along, briefly covering the security aspects of 60ish AWS products.https://t.co/73XBs0vaXU
— Michael Macnair (@michael_macnair) April 8, 2019

Streaming from a USB record player to Sonos speakers via a Raspberry Pi

Sat, 02 Feb 2019 00:00:00 +0000

I was very happy to find two guides to setting up a Raspberry Pi to stream music from a turntable with a USB output, so we could listen to it on our existing Sonos speakers and via a network-connected receiver in a different room.

I adapted the two guides and learned a few things along the way – this is mostly a record for my own reference, but perhaps it will be of help to someone else too.

The changes from coreyk’s and basdp’s guides are roughly:

Used Raspbian Stretch (no changes needed)
Use stock darkice binaries from the repository
darkice init script tweaks
Stream from port 80 so you don’t have to specify a port
Power control from a mobile via a URL

Stock darkice
darkice init script tweaks
Default port
Reboot and shutdown from a mobile

Stock darkice

coreyk compiles darkice from source, to add AAC+ support. I don’t need this, as on a local network the suggestion of doing fixed 320kbps MP3 encoding is fine.

basdp provides their own .deb. This is fragile (what versions of raspbian will it work with?) and there’s no way to tell if it’s trustworthy.

I had no issues with the stock darkice. This meant the only packages I needed to install were darkice and icecast2 (and vim!).

darkice init script tweaks

I followed coreyk’s guide, but instead of deleting the pidfile with rm, you can pass --remove-pidfile to start-stop-daemon when stopping.

To get logging output from darkice, I changed the start invocation to make use of --no-close:

LOGFILE="$LOGDIR/$NAME.log"
...
start-stop-daemon --start --quiet --make-pidfile --pidfile $PIDFILE \
            --background --chuid $USER:$GROUP --no-close \
	    --exec $DAEMON -- $DAEMON_OPTS >> $LOGFILE 2>&1

Not directly related to the init script, but note that darkice is being run as nobody:nobody. This user can’t set the realtime scheduling priority requests in darkice’s configuration file, so we give the binary that capability:

sudo setcap cap_sys_nice=+ep `which darkice`

Default port

Both guides suggest setting up icecast to listen on port 8000. This is fine, but makes the URL slightly ugly.

With port 80, you can stream from http://vinyl.local/listen.

The port can easily be changed in /etc/darkice.cfg and /etc/icecast2/icecast.xml, the trick is to give icecast the capability to bind to port 80:

sudo setcap 'cap_net_bind_service=+ep' `which icecast2`

Reboot and shutdown from a mobile

To provide a clean shutdown option, I wrote this little script to listen for web requests. A shortcut to the URL alongside the Sonos app on my phone’s launcher then provides a convenient way to cleanly shut it down. I don’t know what the risk of SD card corruption is from hard power-offs, but this eases my mind.

It’s called from /etc/rc.local so it should always be running when the Pi is. This also means it runs as root - not generally a good idea for a web server, to say the least, but in this case it’s quite convenient as that gives it permission to shutdown and reboot.

#!/usr/bin/env python3

from http.server import BaseHTTPRequestHandler, HTTPServer
from urllib import parse
import subprocess

# HTTPRequestHandler class
class handler(BaseHTTPRequestHandler):
    def do_GET(self):
        parsed_path = parse.urlparse(self.path).path
        if parsed_path in ['/reboot','/restart']:
            code = 200
            message = "Rebooting..."
            subprocess.Popen(["shutdown", "-r", "now"]) # will run concurrently
        elif parsed_path in ['/halt','/shutdown','/stop']:
            code = 200
            message = "Shutting down..."
            subprocess.Popen(["shutdown", "-h", "now"]) # will run concurrently
        else:
            code = 404
            message = "Not found. Available paths: /reboot, /halt\n"

        self.send_response(code)
        self.send_header('Content-type','text/html')
        self.end_headers()

        # Send response to client as utf8
        self.wfile.write(bytes(message, "utf8"))
        return

def run():
    # bind to all addresses, unprivileged port
    server_address = ('', 8000)
    httpd = HTTPServer(server_address, handler)
    print('running server...')
    httpd.serve_forever()

run()

Patching node packages

Fri, 09 Feb 2018 00:00:00 +0000

An investigation into managing patches for node libraries.

I’m developing an electron application, and whilst using a third party library came across a missing feature that upstream don’t want to implement. It can be implemented with a trivial one-line change, but I found the real difficulty was working out how to manage this patch. This isn’t an uncommon scenario, but I’d found very few people talking about how to manage such situations.

This post goes over the various options I considered and ultimately makes a recommendation on what approach to use when.

Local copy

You could copy the entire module into your project, and commit it with your fix. Gross.

Fork

You could fork the module and make the changes in the fork. npm supports installing a module directly from a git repo, so that’s straightforward. This is the usual approach to this problem and in many cases is probably the best solution.

The downside is tools will no longer detect your dependency: say goodbye to npm, greenkeeper, or snyk notifications about updates or vulnerabilities in that module, for example. I’d quite like to have my cake and eat it too – I want to track upstream without having to remember to do something different for a special subset of dependencies.

post-install patch

npm will run the postinstall script if it’s specified in package.json. You could use this to apply a patch to the installed module. At first glance this is neat – it happens at the right time, and requires minimal maintenance – just add a patch file to the repo and tweak package.json.

One hiccup is the availability of patch – it’s a unix command. Fortunately git exposes patch functionality under git apply and any development platform will have git installed if your application is managed in git.

However: whilst this will work for the first npm install, subsequent invocations (for example after updating any other module, or in a CI system that caches node_modules) will try and apply the patch again to the already-patched module, which will fail because patches aren’t idempotent.

You could work around this by making the script leave a marker behind, say adding a .<myproject>.patched file in the module’s root or similar. This loses some of the cleanliness of the approach, but it could work.

Update: it turns out about a year before I wrote this patch-package was released which uses this approach in a very nice manner, and I just didn’t find it when doing my research. It allows you to make the fix directly in node_modules, and will then generate the patch for you and apply it to future installs.

There’s a module for that

node_module_patches gets around the idempotence issue by overwriting entire files instead of applying a standard patch. You set up a mirror-hierarchy of node_modules under node_modules_patches, any files in there are copied over the top of the original ones, and it displays a diff for the user. You could (should?) run this from postinstall as well.

This might be a reasonable halfway-house, but I don’t like the “commit a random file from another project into your repo” and I really don’t like that you could totally break your dependency and not notice. Because it’s not a proper patch, the module could change in any way and this will happily overwrite the file. It’s on the user to decide that the diff doesn’t look right any more. Yuk.

Build system patch

A tool like grunt, gulp, or webpack could implement the same behaviour as the postinstall script described above by having one of the build steps apply a patch. I see two options: directly patch the installed module in node_modules, or exploit node’s search behaviour to create a (patched) copy of the module. Directly patching the installed module has the same issue as using post-intstall in that patches aren’t idempotent.

The latter option is more promising. In my case, I’m already using gulp to build my code; the output goes into an app directory. If the patched package is placed under app/node_modules/ it will get picked up by node in preference to the original in the application root.

The approach I’ve taken is to create a patches directory, which contains a directory for each package that needs patching. These directories contain one or more patch files that will be applied to that package.

The gulp-apply-patch plugin fits the bill nicely for this task. Here’s the relevant part of my gulpfile, I’m fairly happy with how simple it is:

const patch = require("gulp-apply-patch");

const BASE_DIR = path.resolve(".");
const BUILD_DIR = path.join(BASE_DIR, "app");
const PATCHES_DIR = path.join(BASE_DIR, "patches");
const NODE_MODULES = path.join(BASE_DIR, "node_modules");
const PATCHED_MODULES_DIR = path.join(BUILD_DIR, "node_modules");

gulp.task("patch", function () {
  // Get all the directories in the patches folder
  let to_patch = fs
    .readdirSync(PATCHES_DIR)
    .filter((f) => fs.statSync(path.join(PATCHES_DIR, f)).isDirectory());

  // If there's only a single entry in the patches directory, the {set glob}
  // syntax won't match, so add a dummy entry to work around that
  to_patch.push("nonexistent_dummy_package");

  // Convert the patch directories into a glob that matches all the files
  // from the corersponding source packages.
  let globs_to_patch = NODE_MODULES + "/{" + to_patch.join(",") + "}/**/*";

  return gulp
    .src(globs_to_patch, { base: NODE_MODULES }) // copy only the modules that have patches
    .pipe(patch(PATCHES_DIR + "/**/*.patch"))
    .pipe(gulp.dest(PATCHED_MODULES_DIR));
});

This has some nice properties:

The change being made is explicit, as there’s a patch file in the repo.
When upstream changes, either the patch will continue to apply and all is probably well (diff line numbers and context lines haven’t changed), or it will fail to apply and you know to check and update.
From the perspective of tools looking at package.json, you’re using the standard upstream library. Everything works normally.

The main downside compared to the fork approach is maintaining a patch file is more work compared to maintaining a forked repo. You can’t just git pull, unless you combine both approaches and generate the patch from a fork. It is also slightly higher risk, as after an upstream change the patch might still successfully apply, but it now breaks the package in a way that would have been picked up had you run the package’s tests after pulling the changes into a forked repo.

Monkey patch

Thanks to the wonder that is javascript we could use the original module unmodified, but at run-time try and fix it up to have the behaviour we want. Depending on the patch this could be trivial and relatively clean – perhaps an ‘internal’ integer should have a different value. It could be really ugly if, say, a large function needs a slight logic change – the whole function will need to be replaced, implying your patching code has to have a complete copy of said function.

/_ eslint-disable _/
// monkey patch to make it do the right thing
originalModule.func = function() {
...

Good luck getting that to type-check in TypeScript!

This feels fragile, kind of like node_module_patches – you might well only know when it breaks when you (or your users) hit the bugs at run time.

Other languages

How is this solved in other languages?

Composer is a package manager for php, partly inspired by npm. There is a plugin for composer called composer-patches which does exactly what I’m looking for: it applies a patch to dependencies as they are installed by composer. Because it’s integrated with the package manager, it avoids the issue with using npm’s postinstall script described previously – the patches are only applied when the module is installed. It goes further than this and supports dependencies applying patches to their own dependenciesto their own dependencies – this seems a little problematic, as it encourages creating libraries that need patches to be applied, which in turn forces any users of your library to use composer-patches as well. Neither npm nor yarn support plugins.

Go has some particular problems with the fork approach, because dependencies include their full path: import "github.com/a/b". These can be worked around, but that’s all I’ve found on this topic.

The tradition in ruby seems to be monkey patching.

I couldn’t see anything beyond the fork approach mentioned for python or rust.

Conclusion

Forking is the most robust approach, and you should probably do this. However if you’d like your existing dependency management tools to continue working with this patched dependency, there are a few options.

Update: you should probably just use patch-package, a robust install-time patching tool.

It may be worth considering monkey patching - is your change small, easy to implement at runtime, and when upstream changes likely to either be robust or fail noisily? This could be the simplest approach.

In preference to monkey patching I would probably recommend applying a source code patch as part of a build step. It’s relatively robust, easy to manage, and less fiddly than trying to monkey patch.

Hopefully this will save someone the time I spent searching for something like this post! If you have any thoughts on the subject, or decide to re-implement my solution as a gulp plugin, please let me know!

afl-fuzz on ppc

Wed, 27 Apr 2016 00:00:00 +0100

afl-fuzz gives you great fuzzer coverage for very little investment of development time. Its primary target platform is Linux on x86 / x86-64, however since the introduction of LLVM mode / afl-clang-fast, the docs (link is to an unofficial mirror) offer some hope for those of us working on less mainstream architectures:

The instrumentation is CPU-independent. At least in principle, you should be able to rely on it to fuzz programs on non-x86 architectures (after building afl-fuzz with AFL_NO_X86=1).

This is, however, the grand total of the documentation available for using afl on other architectures. I’m pleased to report that it works in practice as well as principle, though it’s not straight forward. Here is what I did to start fuzzing with afl when cross compiling for PowerPC targets (ppc & ppc64).

(Update 2017-07-17: minor improvements and a cleaner approach that doesn’t involve installing afl.)

Prerequisites
Aside: cross compilation with clang
Cross compile afl-fuzz and the afl runtime
Usage
Summary

Prerequisites

For this to work you need to be able to compile the software to be fuzzed with vanilla clang and successfully run it on the target. How you do this depends on your software, see below for some difficulties I encountered.

Then you need to compile the afl LLVM mode compilers in the usual fashion for your host system, we’re going to put it in a directory called afl-cross:

wget http://lcamtuf.coredump.cx/afl/releases/afl-latest.tgz
tar xzf afl-latest.tgz
mv afl-2.46b afl-cross
cd afl-cross
make && make -C llvm_mode

Aside: cross compilation with clang

This isn’t afl specific, but is another problem I needed to solve before I could use afl’s LLVM mode compiler.

On the one hand, cross compiling with clang is a breeze because it has built in support for it - you don’t need to compile a dedicated cross-compiler for the job. On the other hand….

With gcc cross compilation we normally specify two different paths - the sysroot for the target headers, and the path to the host’s cross compilation tools. clang doesn’t support this concept properly, having only the concept of the sysroot where it expects everything to be. As clang doesn’t have full support for cross compiling to ppc, it still needs to use your cross-tools version of ld and as. To fix this you can use the undocumented -B switch which instructs clang to search for its guts there first, before its standard paths. Very unsatisfactory, but it does work. You get an invocation that looks like this:

clang -target magic-target-triple -B /path/to/cross/tools/dir -mcpu=<mycpu> \
 --sysroot=/path/to/sysroot -o myExec myExec.c

Actually working out the correct magic-target-triple was far harder than it should have been. Hopefully these improvements to clang will be implemented to help you enumerate the supported targets - until then, searching + guessing was all I had. 32bit ppc target triples seem to take the form powerpc-<vendor>-linux.

If you’re cross compiling for a target that has full support in LLVM then this should be a lot easier.

Cross compile afl-fuzz and the afl runtime

Now we need two things that are compiled to run on the ppc target: afl-fuzz (and any other utilities like afl-analyze); and the tiny blob of afl code that afl-clang-fast will add to every object you compile - afl-llvm-rt.o (and potentially its 64bit companion). Unpack another copy of afl into a new directory, let’s call it afl-target. The appropriate invocation will look something like:

tar xzf afl-latest.tgz
mv afl-2.46b afl-target
cd afl-target
( export CC="clang" CFLAGS="-O3 -funroll-loops -target magic-target-triple \
-m32 -mcpu=<mycpu> --sysroot=/path/to/my/sysroot -B /path/to/cross/tools/dir" \
AFL_NO_X86=1 make && make -C llvm_mode )

(I’m using export to avoid repeating CFLAGS for each make command, and a ( subshell ) so the export doesn’t taint the parent.)

Note that the final step in the Makefile is to run a test by compiling something and executing it - this will fail, because the host architecture is different to the target. That’s fine.

Now replace the cross-compiler’s version of this object with the target one, so our cross compiling afl-clang-fast injects the right code:

cp afl-target/afl-llvm-rt* afl-cross/

Usage

At this point everything just works. Cross compile your target software, e.g. with CC="/path/to/afl-cross/afl-clang-fast" CFLAGS="<all-the-flags>" ./configure && make; copy the instrumented binaries to the target; copy afl-fuzz from the afl-target directory to the target; and fuzz in the normal way.

Summary

You can use afl on non-x86 targets, but need to (a) be able to cross compile for your target with clang, and (b) compile the target architecture variant of afl-llvm-rt*.o for use by the host version of afl-clang-fast. afl-fuzz and other utilities compiled for the target can then be copied on to the target, enabling you to start torturing non-x86 software.