According to Wikipedia smart contracts are a computer protocol intended to digitally facilitate, verify, or enforce the negotiation or performance of a contract. Smart contracts allow the performance of credible transactions without third parties. These transactions are trackable and irreversible.
But the question here is that ‘do smart contracts really work?’, especially when they are put into practice in the cryptosphere. Let’s have a more detailed look below.
Looking at the code paths, let’s start with something easy, we will call it ‘Application A’ that can multiply and divide two given numbers, represented in figure 1.
Now let’s think about the possible scenarios this application might fail in. Alternatively you can think about the number of ways to navigate through the application’s code, or the number of code paths in short.
You might say that there are two: one scenario where the user selected the multiply operation and the other where the divide operation was selected. But then you are also saying that you are sure the application will continue to run flawlessly after both operations have been successfully tested once. However, this isn’t necessarily true. With input numbers 5 and 4 being x and y respectively it might execute as expected for both operations. But with numbers 5 and 0 it might not for the divide operation, if division by zero wasn’t properly covered in the code.
So in this example, if numbers x and y were 8-bit numbers, there would be at least 131072 possible code paths, or 256 times 256 times 2 (figure 2). Meaning that, in theory, in each of these code paths there could be a flaw.
smart contracts in crypto
Figure 1 – Application A
why smart contracts do not work
Figure 2 – Possible code paths in application A
The probability of flaws in application code
For a given application we can formulate the probability of (undiscovered) flaws as follows:
smart contracts equation
Where P(F) is the resulting probability, U the number of untested code paths, C the possible number of code paths and P(Fu) the probability of a flaw in an untested code path. The latter is theoretically always 50 percent because, well, it’s not tested. In practice, however, this greatly depends on the quality of the developer (team).
Alternatively the following formula produces mentioned probability after a specific number of code path executions:
Where P(Fe) is the resulting probability after e executions, and e the number of executions. Note that this formula applies to applications where each code path has an equal probability of being executed, like it is the case for our sample application A.
Using the formula we get the following results with regard to application A.
Figure 3 – Probability of flaws in Application A
The yellow line in the graph represents the probability of encountering an untested code path after e executions. The blue and greens lines represent the probability of encountering an undiscovered flaw after e executions, where P(Fu) is 50 percent and 10 percent respectively.
Interestingly enough, if application A was written by a developer who delivers 90 percent correct code on average (P(Fu) = 10 percent), there would still be a 5 percent chance of discovering a flaw even after 80000 application executions, or a 1 percent chance after 300000 executions. If a thousand people made use of this application once a day, it would take 80 days to reach this 5 percent and almost a year to reach the 1 percent.
More concrete, if division by zero wasn’t properly covered upon release of the application, there would be a 50 percent chance that this flaw would remain hidden after 350 executions, and a 5 percent chance after 1500 executions.
And yet we are talking here about the simplest of applications.
Smart contracts in application
Let it be clear that (structural) testing is of the essence before releasing a new contract, or any other application where value is at stake. Testing to the maximum extent possible, that is. As we saw, the possible number of code paths depends on the number of variables an application or code block uses (a smart contract is actually a code block in the greater Ethereum application). More specifically, it depends on the number of possible variable values. Let those variables be blocked variables, application variables or external variables. Even asynchronous execution of the same code should be considered a different code path.
In practice, it isn’t always feasible to test each and every possible code path. We should however test with carefully selected values for each variable and their possible permutations.
After release, the contract has yet to stand the test of time before it can be considered safe, and before any real value should be put in. Generally, the amount of time is dependent on the complexity of the contract.
Equipping smart contracts with an indicator telling the user how safe or smart they actually are, could help avoiding catastrophic failures. Some proof of intelligence, if you will.
Originally posted as ‘Why The DAO didn’t stand a chance’, September 2016
