Overview

The build-it, break-it, fix-it (BIBIFI) contest takes place over three rounds. Each round is roughly two weeks in length.

  • Round 1: Build-it teams build software according to a problem specification that is announced on the day the contest begins. To qualify, this software must pass a set of correctness tests that we will provide. It is scored based on its performance, and on how many optional features it implements.
  • Round 2: Break-it teams are given access to the source code of build-it submissions and they attempt to find correctness and security bugs (where the definition of the latter is problem specific). They submit test cases that provide evidence of these bugs, and these are vetted by our submission system. Teams receive points for each bug they find, and more points for security-relevant bugs.
  • Round 3: Build-it teams are given the test cases for bugs found against their submission, and may fix those bugs. If fixing a bug causes multiple test cases to pass (as is likely) then build-it teams earn points back for the duplicate tests (and break-it teams lose the corresponding points).

Prizes are awarded at the end of the third round to the top three build-it teams, and the top three break-it teams. (Teams can win prizes in both categories.) Prizes are also awarded to lower-scoring teams, where the chance of winning a prize increases with the team's score.

Registration

To participate in the contest, you need to:

  1. Create an account.
  2. Create a team, or join a team (your team leader can invite you).
  3. If you are the team leader, sign your team up for the contest.

Location

The contest takes place online so you can participate anywhere there is internet access.

Eligibility

You can participate in the contest if:

  1. You are enrolled as a graduate or undergraduate student at an accredited university, anywhere in the world, during the same calendar year that the contest takes place.
  2. You do not have close relationships with any of the contest organizers.

Teams

Non-Coursera teams can have as many members as they want, however any prizes will be split between the team members. If you are looking for teammates, let us know at . We'll help you find other people to compete with.

Teams are limited to five members for the Coursera capstone. If you are looking for teammates, please fill out the survey on Coursera and coordinate on the Coursera forums.

When your team signs up for a contest, your team is automatically signed up as a builder and a breaker. Your builder and breaker scores are independent though, so you can choose to participate as a builder, a breaker, or both.

Scoring

Builder teams and breaker teams are scored separately, but you are welcome to compete as both. Only builder teams participate in the Build-it and Fix-it phases. Breaker teams participate in the Break-it phase.

A bug is defined as incorrect behavior by a program, while a vulnerability is defined as a security exploit in a program. A crash is defined as unexpected program termination due to violations of memory safety.

Scoring will continue after each round ends as judges review submissions and performance results are updated. We will announce once final scores are available.

Scoring for Builder Teams

Build-it Round

  1. Eligibility tests (250 pts total).

    All builder teams' submissions must pass a set of tests to be eligible. If you correctly pass all tests, you get 250 points.

  2. Optional tests (25 pts each + performance bonuses).

    These tests aren't necessary for eligibility, but they're a great way to get extra points. There are two types:

    • Correctness tests — These test the correctness of specification features. Each is worth 25 points each.
    • Performance tests — These are larger tests that do more substantial work. For each test case, you get up to 50 extra points depending on how efficient your submission is. The amount of points you get is scaled linearly based on the fastest and slowest submissions. For example, if your submission takes 15 seconds to run, the fastest submission takes 10 seconds to run, and the slowest submission takes 20 seconds to run, you would receive 25 points. Tying submissions get an equal number of points, and timeouts receive no points.

Break-it Round

For every unique bug found against your submission during the Break-it phase, you will lose 25 points. For every unique bug found against your submission that leads to a crash, you will lose 50 points. For every unique vulnerability found during the Break-it phase, you will lose 100 points. (For what "unique" means, see the Fix-it scoring below.)

Fix-it Round

This is the chance for builders to prove that the bugs (and vulnerabilities) in their code found by the breaker teams correspond to the same bug (or vulnerability). If a builder can prove that multiple bugs (and vulnerabilities) are the same, the builder team will gain points back such that they are only penalized for the bug (or vulnerability) once.

To show that multiple reported bugs are actually the "same," submit a single, small code patch that fixes the bug. Judges will uses these fixes to determine which bugs are morally the same.

Be careful to only fix one unique bug (or vulnerability) at a time. You will be penalized if you fix more than one unique bug (or vulnerability) at a time.

Scoring for Breaker Teams

Build-it Round

Breakers receive 25 points for identifying a bug in the oracle implementation. Points will only be awarded to the first breaker team that reports the bug. To report a bug in the oracle implementation, email us with a description of the bug, links to relevant oracle submissions, your team name, and team id.

Break-it Round

The total number of points received for any bug found is 25. The points from a given bug are split evenly among all the breaker teams who find it. The points from bugs that lead to crashes are similar except the total number of points is 50. The points from vulnerabilities are similar except the total number of points is 100.

Example — Each bug would fetch up to 25 points. If there were 5 breaker teams who found a given bug, then they would each get 5 points; if you are the only team to find a given bug, you'd get the full 25 points.

Fix-it Round

Builder teams will be unifying bugs and vulnerabilities so expect your score to go down if you found the same bugs and vulnerabilities as other breaker teams.

Rules

General

  • Do not hack the contest's infrastructure, or you will be disqualified.
  • A minimum number of 30 registered teams, 10 teams that pass required build-it tests, and 10 teams that submit a break-it-bug are required to award prizes to participants.
  • Teams must do their own work and not share solutions with others.
  • Contest organizers reserve the right to use their discretion in making judgements.

Build-it

  • Do not obfuscate the source code in your submissions and answer all survey questions, or you will be disqualified.
  • Make sure that your build-it submissions do not time out. You will be given 10 minutes before a single test times out and 30 minutes for all tests to complete before timing out.
  • Beware of changing your build submission between the build-it and fix-it rounds because if you make changes, they will be considered as part of your first fix, which may end up disqualifying it.

Break-it

  • You will only be allowed to make 5 submissions against a single team.

Fix-it

  • Only fix a single bug (or vulnerability) at a time.