I'm a Berkeley prof. Working at a startup led me to update my 1500-person class

I’m a teaching professor at UC Berkeley in the department of Electrical Engineering and Computer Sciences, though recently I’ve been devoting some of my time to helping build a startup called PerfectRec. As this is my first industry experience, I’ve learned a ton, and it’s got me thinking about how I might bring some of the principles I’ve learned into my classes. 

The course I teach most often is a 1,500-student data structures class that’s typically taken by freshmen and sophomores. In addition to CS fundamentals, this course provides a basic introduction to software engineering ideas such as unit testing, test driven development, and version control. One big omission is guidance on team workflows. Student approaches are typically quite naive, e.g. team members push to their main branch, with no code review or testing required.

At PerfectRec, I’ve seen better ways to do things, having been the head of machine learning, a data scientist, and more recently a software engineer. We’re building a recommendation system, first focusing on consumer electronics such as TVs, phones, and laptops. We ask users multiple-choice questions and try to come up with the perfect suggestion. The long-term plan is to be able to recommend anything, even answers to bigger life questions, like where you should live or go to college. Fun! But getting people coordinated across three teams (web dev, ML dev, human experts) is challenging, especially at a fully remote company.

Which is why I’ve found the CI/CD tools we use at PerfectRec to be inspiring. I’ve started to brainstorm a course assignment that would include code review and Github Actions to demonstrate the power of CI by having all 1,500 students contribute to a single GitHub repo. Here’s an outline of the idea which I’m tentatively calling “The Lottery in Babylon” (a la Borges), which I’d be rolling out in the next academic year.

In the Lottery, each student will select a unique integer as their champion, giving us ~1,500 unique entries. We’ll hold an 11 round, single elimination tournament of one-on-one matches until only one integer remains. Each of the matches will be judged using a randomly selected Comparator from a pool of 1,500 different comparators. In the case of a tie, we’ll hold a repeat match with another random comparator. An example is given below for the case of 4 entrants rather than the 1500 that we’ll use in the real assignment:

The Comparators will come from the students, each contributing their class to the project repository. Every student will write a class called JudgeXXX.java, where XXX is their public student ID number. Each Judge class will implement Comparator<Integer>. Their judge may do anything they want, so long as it is at least somewhat interesting and creates a deterministic total order, i.e. obeys what Knuth calls the “law of trichotomy” (exactly one of a < b, a = b, and a > b is true) and “law of transitivity” (if a < b and b < c, then a < c). 

For example, one student might decide to implement a Comparator that compares two numbers based on how many 1s are in the number, e.g. by this example 11851 is less than 1111. Another might count the number of pencil “strokes” they think it takes to write the number in their own handwriting, e.g. 660 < 44. These Comparators are entirely up to the students so long as they result in a total order.

The chance of any student’s integer encountering their own Comparator is low, so we’re hoping students will think creatively rather than just making their number the “biggest” by their ordering. One open question is how to encourage more creativity.

Every student will create their own branch and submit a pull request for their Comparator to the repo that will be used to run the lottery. We’ll use a Github Actions workflow to verify that their Comparator creates a total order.

To have their pull request approved, every student will have to have their code reviewed and approved by at least 3 reviewers, who will verify that the javadocs for the Comparator match the Comparator’s behavior. Every student will have to approve at least 3 other student’s submissions. We’ll set things up so that they can’t modify any of the code used to run the contest, though I'm open to cool ideas about how we can potentially allow this.

Once we have all of the comparators and unique integers, we’ll run the tournament and find the winning integer. I haven’t yet thought of a suitable prize other than the satisfaction of knowing that you won against the odds in an infinite game of chance. 

While this is a bit artificial, I think it’ll give a glimpse at how CI practices can be used to coordinate the actions of a vast number of developers. We can later build upon this experience and provide guidance for leveraging these practices in small teams for the open-ended capstone project, but that’s a post for another time.

Thanks to my former TAs Dominic Conricode, Stella Kaval, Noah Adhikari, and Samuel Berkun for feedback on this idea, especially Samuel, who had the idea of getting every student in the class working on one big repo.