Oct 022020
 

Planning retirement finances is harrowing; finding reliable, disinterested information is difficult. This paper studies and constructs a portfolio based on a set of formal requirements and analyses its potential performance by back-testing on 20 years of history.

Problem statement and Requirements

Given a single down-payment, construct a portfolio that will generate a retirement pension.
A stipend will be withdrawn the beginning of each year. The amount will increase by 1.3% p/a to compensate for the average inflation in Switzerland.
The portfolio’s life shall exceed 30 years.

Objectives

  • The risk of total loss is minimised.
  • The portfolio shall be as resilient as possible to market turmoil.

Constraints

  • Instruments. Given the problem statement, the only viable instrument is shares.
    • Bonds cannot produce the required revenue.
    • Derivatives are subject to default by the writer.
    • Funds have a risk of default by the managing entity and their performance is predicated by fees.
    • Real-estate companies (as opposed to funds) that actually possess all their properties are acceptable because they are backed with physical property.
  • Currency. CHF only; forex risks are unacceptable because every currency has lost value against the Swiss Franc in the last 20 years.
  • Geography. The shares shall be in Swiss companies domiciled in Switzerland.
  • Debtor Diversity: No single company shall represent more than 5% of the portfolio.
  • Industry Diversity: No single industry shall represent more than 5% of the portfolio.

Deliverables

  • A set of objective criteria for stock picking that fulfils the constraints.
  • Based on these criteria, a basket of desirable shares in which to invest.
  • Buying, re-balancing and selling strategies.
  • An estimate of the stipend percentage range.
  • An analysis of the basket’s performance, by back-testing on historical data 2000-2019, which will determine:
    • The buffer that needs to be kept to maintain yearly payments, so as not to be obliged to sell at an unfavourable moment.
    • The portfolio’s probable behaviour in worst-case scenarios (e.g. entering the market just before a stock-market crash).
    • Situations in which the portfolio’s life might be compromised and mitigating actions.

Selection Criteria

An ideal company would have a healthy balance sheet, exhibit constant growth and lose no value during a stock-market crash. Such companies do not exist, however amongst the 40-odd financial ratios available, 3 are particularly useful to filter out desirable candidates.

  • The Sortino ratio, a downside variant of the Sharpe ratio, is a strong predictor of companies that have the desired kind of behaviour.
  • The Quick Ratio, whilst dependent on the industry, generally demonstrates that a company has had the foresight to keep enough cash on hand to weather difficulties.
  • Resistance to turmoil, which can be assessed by observing the stock’s behaviour during a stock-market crash. It consists of two associated components:
    • Resilience. The percentage value remaining after a crash.
    • Recovery. The period of time before the stock re-attains its pre-crash value.

Graphically:

Ideal shares will have a resilience close to 100% and a short recovery.

(This graph is in fact the SPI and there were two major crashes. The latter is used as it is more recent. Thus, the SPI has a resilience of 47% and a recovery of 23 months).

Stock Selection

An initial selection is performed where companies will be scored using a decision matrix with 4 components outlined above:

  • The Sortino ratio, weight 20%.
  • The Resilience, weight 30%.
  • The Recovery, weight 30%.
  • The Return (growth plus dividend), weight 20%.

Note: The gold standard for expressing portfolio returns is the CAGR (Compound Average Growth Rate) but as it is calculated using solely a start and ending value, it is sensitive to the end-points:

This extreme case has a CAGR of -0.1% (the blue line) but over the entire period its global trend is positive (the green line, a least-squares linear fit).

Shares’ growths will be measured using the slope of the green line, expressed as a percentage of the average value, in this example 12%. The CAGR will also be calculated and used as a cross-check for cases like this.

From this subset a manual review will determine the companies finally chosen (judgement is required for the Quick Ratio, which has different meanings for each industry).

Universe

The initial instrument universe is the SPI and Swiss Real-Estate companies. Of the 215 SPI shares, 205 are quoted in CHF and totally domiciled in Switzerland. Of these, 82 and have been quoted continuously since 2000 and show positive returns.

Duplicate shares (e.g. Lindt N and Lindt PS) are be eliminated by the lowest score.

To meet the diversity constraint, a single stock with the best score of each industry segment is retained.

Selected shares

After applying the criteria and manual selection 21 shares remain:

These will constitute the proposed portfolio. The benchmark is the SPI.

Rejects

The remaining shares were eliminated because:

  • Railways. Despite their high scores, TITL BN BERG and JUNGFRAUBAHN HLD both have their entire infrastructure in one location. They were dropped in favour of BVZ HOLDING, which is significantly more diversified.
  • Foods. NESTLE scored lower but was preferred over BELL AG, which is not entirely Swiss and focussed on a single product line, meat. GROUP MINOTERIES has poor returns.
  • Chocolates. BARRY CALLEBAUT has a slight edge over LINDT and VILLARS.
  • The remainder due to insufficient resilience and/or recovery.

Weighting

The SPI index components are weighted by companies’ market value, as the aim is to assess the performance of the total CHF amount invested in the market.

On the other hand, an investor’s components are chosen to diversify risk; it is thus undesirable to give more weight to any particular company as this would proportionally increase risk for that component. Consequently, the portfolio will be built with equal CHF amounts of each stock.

Performance Analysis

The baseline comparison consists of comparing the portfolio against the SPI and SMI, normalised to 100 on the 1st of January 2000:

Despite the impression due to the scale, all three lost ~55% in the 2007-2008 crash.

The portfolio recovers much faster (20 months), than the SPI (50 months) or the SMI (73 months).

Note: The SPI’s CAGR over the last 30 years is 9.89%.

Buffering

Selling shares to obtain cash for stipends is undesirable after a stock market crash, as their value will be down by some 55%. There is thus a requirement for a cash buffer to avoid such sales.

The portfolio recovers from a crash in some 20 months. Assuming that a stock-market crash occurs about once a decade, there will be two or three stock market crashes during the portfolio’s lifetime.

As the first crash can occur at inception, the minimum buffer is 2 year’s stipends. Assuming that the stipend is low enough to allow the portfolio to grow during the first half of its life, it may not be necessary to buffer for subsequent crashes.

Buffering with cash is costly with the current -0.7% negative interest. An attractive alternative is to use Real-Estate, which offers more modest growth but suffers much less in troubled markets. Of the 31 Real-Estate companies quoted on SWX, only 5 meet the criteria:

Their worst-case behaviour is a loss of ~15% and a recovery of 11 months, which is an acceptable level of risk given their average growth of 6.6%. (A 15% loss is covered by growth if the first crash occurs no earlier than 27 months after inception.)

This buffering strategy will be tested in the simulation.

Buying, Re-balancing and Selling strategies

Buying

Of the 116’213 price changes of the members of the portfolio in the past 20 years, the average change was +0.0056 CHF. It follows that given the choice of buying today or waiting until tomorrow in the hope that the price will go down, it is logical to buy today (with an appropriate limit to avoid getting executed ‘high’ in the day).

This can by improved by observing a stock’s historical random walk to determine the chances of buying at a lower price over some period. For a given stock, for each day in the history, determine how often a price fall of at least X% occurred in the N following days (here, N=60):

Thus, if I want to buy Barry Callebaut, there is a 72% chance that it will fall by 2.5% in the next 60 days. Intuitively, sought-after shares (Givaudin, IVF Hartmann and Nestlé) are less likely to fall, so the limited order will need to be closer to the current price. Conversely, more volatile shares (Straumann and Tecan) are more likely to be had at a larger discount.

The buying strategy is thus to place limited orders at a discount of the percentage which meets the chosen chance of success. I implemented this strategy on the 2nd of October 2020. On the 20th of November, I had acquired 19 out of the 20 stocks at an average discount of 2.3% below the October 2nd prices, so this strategy works (except for Kühne & Nagel, which never fell 3% during the period).

Re-Balancing

The average cost of a transaction is about 0.85%, so the re-balancing cost is 1.7% of the position. This churning must be weighed against the hypothetical gain obtained by improved diversification.

It makes more sense to sell smaller fractions of multiple positions rather than entire positions, as the difference in transaction fees is negligible (0.825% for transactions over 50’000 versus 0.9% for those below 50’000).

Selling

The stipend pay-out will be made each year, using the dividends and sales as necessary. The simulations determined that the optimal rules for selling are:

  • When the market has fallen over the preceding year, prefer to sell real-estate otherwise sell shares (avoid selling shares at a poor price).
  • Select the 5 shares that have the highest growth in the last 12 months (5 was determined empirically).
  • Choose the two largest CHF positions.
  • Sell half the necessary amount of each of the two positions (to minimise unbalancing).

If a sale would represent more than 90% of the position, sell the entire position (to avoid subsequent odd-lot trades.

Simulation

A simulator was built to perform the back-testing. It operates as follows:

The simulation is initiated by:

  • Setting the clock to the start date.
  • Depositing CHF 1’000’000 in the account.
  • Immediately paying the stipend, proportional to the time remaining in the current year.
  • Use the remaining cash to populate the portfolio with shares and real-estate.

Then, repeatedly:

  • Advance the clock to 31st of December.
  • Calculate the dividends from the current positions and remove 35% withholding tax (which is written off). Credit the portfolio cash with the remaining 65%.
  • Pay-out custodian fees of 0.35% of the portfolio’s current market value.
  • Advance to 1st of January.
  • Increase the stipend by the rate of inflation.
  • Sell a sufficient amount of positions to make the cash greater than the current stipend.
  • Pay-out the stipend (or declare failure if there is insufficient cash).

Finally, calculate the P&L and various statistics.

Transaction fees are applied at the rates for UBS Online Banking. The rate varies by transaction size, about 0.85%. Other brokers / custodians charge significantly less.

SWX fees and Stamp Duty are also deducted.

Results

A typical simulation produces the following:

Discussion

This is a worst-case, the 2000 stock-market crash commenced on the 25th of September 2000. The other worst case, starting on the 15th of October 2007, produces almost identical results.

The stipend is 7.2%, CHF 72’000 on the first year, increasing to 92’026 after 20 years.

There are practically no sales at a loss.

The portfolio’s costs are 95’606 ÷ 20 years = 4’780 per year = ~0.5%. This could be significantly reduced by using a less expensive broker / custodian, e.g. SwissQuote.

The total stipends amount to 1’632’512 for an initial investment of 1’000’000.

The portfolio’s final value is just over double the initial deposit.

The portfolio has a CAGR of 3.8%, despite the stipend withdrawals.

The asset allocation remains moderately well balanced at the end of the simulation, the 21 initial companies all remain.

Determining the ideal buffer

Simulations were performed for varying buffer sizes from 0 to 6 years. Whilst increasing the buffer has a positive effect on the portfolio’s lowest value, the final portfolio value and the CAGR are affected negatively. It transpires that a buffer of ~30 months is optimal:

Maximum stipend

The portfolio fails (again, starting on worst-case 25/9/2000) at a stipend of 9.1%:

Ideal stipend

The chosen stipend is a function of the investor’s appetite for risk. With a stipend of 7.2% (the typical simulation above), the portfolio final value was 2’056’468 with a CAGR of 3.8%.

Decreasing the stipend to a more conservative 6% increases the final value to well over 3’000’000 with a CAGR of 6.2%:

Observe the transactions after the 2000 stock-market crash, where real-estate buffer is sold rather than stocks, until 2003 with the Givaudan sale.

Unfavourable situations

It is well-known that past performance is not a predictor for the future and a simultaneous stock-market and real-estate crash remains possible. Unforeseeable black-swan events will certainly happen in the incoming 30 years.

That said, most of the selected companies are manufacturers that have an established niche, producing tangible goods. All have a healthy balance sheet and have been well-managed for decades; the risk of total loss is thus extremely small.

Conclusion

The designed portfolio, implemented with a buffer of 30 months and a stipend of 6% meets the requirements with minimal risk.

Post-Scriptum

The same portfolio, run with neither stipends nor buffering and re-investing the dividends, starting on the pre-crash worst-case in September 2000, has the following behaviour (a CAGR of 12.5%):

Jun 022020
 

Back in 2011 the banks, many of them in the UK, were caught cheating by incorrect fixing the LIBOR rates between themselves. The British Financial Supervisory Authority was obliged to stop this and decided that the LIBOR would be abolished at the end of 2021 (it would apparently have been too much of a hardship to ask banks to stop fleecing the market immediately).

The Swiss National Bank had to invent an alternative to the LIBOR, which they did by creating the Saron (Swiss Average Rate Overnight), which isn’t based on rates that banks submit, but the true rate observed every day on the open market. These rates are publicly visible, so cheating is impossible (at least until banks find a novel way to cheat).

Said rates have been known since time immemorial and SIX (the Swiss Stock Exchange) provide a spreadsheet to calculate the Saron rate for any period since 1999. The Saron rates look like this:

Saron rates 1999-2020

Since mid-2015 the interest rate on the Swiss Franc has hovered around -0.7%. Notice that this a is a negative rate, which means that if “A” lends money to “B”, then “A” must pay “B” 0.7% for the privilege of lending him the money. Replace “A” with your bank’s name and “B” with your own name and re-read that sentence carefully.

Traditionally, banks have set LIBOR mortgage rates at about 1% over the interbank rate; their margin for the loan, fair enough. For the 1st of January to the 31st of March 2020, the Saron rate was minus 0.7058%. Adding the 1% margin, the mortgage rate should have been -0.7058% + 1% = 0.2942%.

During that period I paid 0.8% for my 3-month LIBOR mortgage. I have thus been over-charged 172% (0.8 is 172% of 0.2942).

(There is a minor, deliberate error in this. Can you spot it?)

Feb 182020
 

Some years ago I was working for a large multi-national corporation. One morning I received an email from the CIO, addressed to the entire company, explaining a botched SAP rollout. The corporate-speak was some of the best that I have ever read; I reproduce it here, unaltered save for anonymisation, with my translation into layman’s English.

On June 1, we reached go-live with Dressing, Sauces and Oils North America (DS&O) on the TC3 instance of our SAP solution.

On June 1st we forced go-live of our half-baked SAP solution on DS&O, simply because we were so over-budget that the only other option would have been to scrap the project.

Since go-live, DS&O has experienced significant issues with the solution that are impacting the business’ operations and financial reporting.

Since go-live, the business has been losing money like a leaking sieve, because our ‘solution’ was a complete and utter disaster.

This is unacceptable for DS&O and for us.

DS&O have said clearly: “Make it work or die”. For us, ill-engineered solutions are the norm.

To be trusted partners, Global IT commits to fully supporting our businesses … so, we will do what’s needed to stabilize performance for DS&O.

SAP consultants will be brought in, at 3’000$ a day, to criticise the implementation and spend a king’s ransom on new servers that will make the problem worse.

To that end, Andrew Brown, our SAP lead and a member of the CIO leadership team (LT), will personally spearhead the effort to review all factors that could be contributing to the instability.

Andrew’s balls are, in theory, on the table to get this working. In reality, at 3’000$ a day, he doesn’t give a flying fuck, and anyway he has a new client ready to shaft just down the road.

This includes looking at business processes, operations, data as well as assessing the actual solution itself.

We will re-hash the miserable initial requirements analysis and lay the blame squarely with the consultants that are no longer with us.

We want to fully understand the root cause to thoroughly address the issue for DS&O and add to important learnings.

I heard ‘root cause’ in a management course and it sounds good here.  ‘important learnings’ is cute too; I’ve been managing IT for 30 years and I still have yet to learn one single lesson from my impressive catalogue of mistakes.

Andrew will be 100% focused on this effort, with members of the CIO LT stepping in to support other areas for which Andrew is responsible.

Andrew will try to attribute blame on his colleagues. If he is successful, I shall fire them, if not I’ll fire him. With a glowing recommendation.

We undertake this initiative with confidence we can stabilize DS&O and make any warranted enhancements to our approach or technology.

‘warranted’ means that we never made any bad choices initially. ‘enhancements to our approach or technology’ means that we might well need to choose a completely new technology, that will triple the budget and push the project back by three years. Until the next fuck-up.

We will work to ensure that the difficulties experienced here are not repeated.

Those of you who have already been through an SAP implementation will see the humour in this statement.

We will share these learnings with our businesses currently preparing for (or in) deployment.

Our bookmaker is currently taking bets on SAP projects at 15 to 1 of failure.

We also will continue to stress the importance of readiness … businesses and functions must undertake needed changes to prepare for the adoption of new processes, a new organizational structure, and more – all of which are required to execute enterprise process strategies.

Given the magnitude and cost of this unholy blunder, it would seem only fair that you accept some of the responsibility. After all, we did talk to you a couple of times before implementation.

What’s critical to understand is the  Leadership Team (CLT) remains as committed as ever to our ERP modernization journey and SAP as our solution.

We take the meaning of ‘dogmatic’ to levels that even a religious zealot couldn’t imagine. Every morning, we recite ‘Our SAP which art in heaven, hallowed be thy name”.

We are not stopping, we are not slowing down … we are moving ahead with our roadmap and the commitments we’ve made.

Yes, we are going to continue, hell-bent, on forcing SAP down your throats, no matter at what cost or damage to the business, because they had the sexiest PowerPoint presentation.

Leaders have confidence in our function and demonstrating our commitment to their businesses (as we are with DS&O) means we will continue to deserve that confidence. 

Truth be told, the business is sick and tired of IT delivering shitty service by some ignoramus in Bangalore.

Should you have questions about our work on DS&O, please reach out to your manager, SAP leadership or your CIO LT member.

For the British, ‘reach out’ has a distasteful innuendo, possibly acceptable in such a gush of platitudes. Notwithstanding, if you have any self-respect, find a job elsewhere.

Thank you,

If I was honest, I’d say I’m sorry, but I’m above that

Brian

Feb 032020
 

Messieurs,

Il est important de protéger l’environnement, donc j’ai commandé des panneaux solaires et une pompe à chaleur, qui me libéreront de ma dépendance aux énergies fossiles et m’éviteront dorénavant de brûler 6 tonnes de mazout chaque année. Il est édifiant de découvrir que cette démarche n’a strictement aucun intérêt financier. 

Le subside que je recevrai de la Confédération n’est en réalité qu’un prêt qui sera remboursé en moins de sept ans avec les impôts que je paierai sur le courant que je vous revendrai.

Le récent fiasco des contrats dénoncés unilatéralement par SwissGrid pose une lumière crue sur les rétributions accordées aux particuliers qui revendent leur électricité. La journée, vous me facturez 26 ct/kWh et vous allez me le racheter 12 ct/kWh, soit un bénéfice de 116%. Cela alors que vous n’apportez quasiment aucune valeur ajoutée à la transaction, car les électrons que je générerai seront utilisés par le consommateur le plus proche : mon voisin.

On serait tenté de croire que votre politique de rétribution est mue par des considérations purement mercantiles, mais celles-ci ne résistent pas à l’analyse : la production privée, presque homéopathique, se mesure en MWh alors que vous traitez en TWh.

La raison réelle est plus sournoise. L’idée qu’un consommateur aie ne serait-ce qu’un peu d’indépendance énergétique vous est totalement rédhibitoire et les restrictions des volumes de fluides caloriporteurs le confirment : vous ne tolérez même pas que je puisse emmagasiner de la chaleur le jour afin de l’utiliser la nuit suivante.

Dès lors, on constate que la sollicitude des instances publiques pour l’énergie renouvelable n’est qu’une fumisterie hypocrite ; celui qui produira de l’électricité verte le paiera intégralement de sa poche et sera taxé pour son impudence.

Le seul espoir reste dans la prochaine ouverture du marché de l’électricité, qui sonnera le glas de votre monopole et peut-être l’arrivée de concurrents plus enclins à acheter une énergie propre à un prix équitable. Cela serait un vrai encouragement à l’abandon des énergies fossiles si nuisibles à notre environnement.

Recevez, Messieurs, l’assurance de mes sentiments distingués.

Maurice Calvert

Jan 082020
 

In order to create an accurate plan of an old house, I purchased a Leica X4. It is a robust, pleasant instrument, clearly designed by engineers, for engineers and the build quality is faultless.

My first pleasant surprise was that each device is comes with an individual calibratation certificate and the tolerances are much tighter than the blurb in the spec-sheet:

As it should be, the deviation tolerance (±2σ) is supplied at a specified temperature, with tolerance (±3°). Second pleasant surprise, 0.2mm at 7.8 metres is 0.002% – this is clearly a laboratory-grade instrument.

Indoor measurements are made with the red laser dot, outdoor measurements can also be made using the builtin zoomable camera, both very intuitive

The quick-start guide is about as terse as can be – essentially useless for more than using the device as a tape measure – you’ll need to download the manual to be able to make use of all the X4’s features; it is well-written and easy to follow. It took me a good half-hour to become capable of making every measure possible, they are extensive:

  • Room area and volume, corner maxima, wall width from 3 points
  • Angles and resulting line projections
  • Indirect object height from measure-to-base and altitude angle (to measure the height of a tree without a laser dot on the top branch)
  • Stake-out
  • Min-max measures
  • Bluetooth connection to Android phone to transmit values
  • … and so forth

In a nutshell, the X4 can measure (or deduce using Pythagoras), almost any measurement you can imagine and it does it quickly and precisely.

The indirect height function was disabled as-delivered, I had to download the firmware update to my Android and flash it to get this to work (I tested it against known targets and it works extremely well). Aside from firmware updates, this “DISTO Plan” program offers several plugins for plans, facades and room layouts. The interface is clumsy and the advanced features are payable. Disappointing.

Verdict: The X4 is a gem of precision Swiss engineering that is a delight to use and of extraordinary accuracy; just delete the Android software once you’ve used it to flash the latest firmware.

Jan 082020
 

Today’s Guardian has this piece https://www.theguardian.com/politics/2020/jan/07/eu-parks-post-brexit-demands-avoid-early-clash-boris-johnson-ursula-von-der-leyen

To save you clicking:

The European parliament will express its “grave concern” about the attitude of Boris Johnson’s government to the 3.3 million EU citizens living in the UK following threats of deportation made by a British minister.

In a leaked resolution drafted by the main political groups and due to be backed by MEPs next Wednesday, Johnson’s administration is accused of creating “anxiety” in recent months.

Dear Ursula von der Leyen: Britain will shortly be leaving the EU and you appear to be unsettled that EU citizens will no longer enjoy freedom of abode in the UK. One of the primary purposes of Brexit was to allow the UK to decide who may and may not settle in the UK, without external interference. You apparently haven’t grasped that we will be able, if we so choose, to treat Belgians and Spaniards no differently to those from Botswana and Taiwan. We are of course more than willing to negotiate preferential terms for Europeans, but negotiation carries no obligations; both you and we can walk away from an unsatisfactory agreement.

To be perfectly clear: We owe you nothing and you owe us nothing. Both our mutual privileges are gone. The sooner you understand this, the better.

Nov 032018
 

Intel’s RealSense cameras are astonishingly precise but not as accurate. By optimising the calibration of the depth stream and correcting for non-linearity, the accuracy can be improved by an order of magnitude at 2.5 metres and becomes almost linear in the depth:

Skip to solution

Sources of error

Calculating the coordinates of the 3D point corresponding to a depth reading is straightforward trigonometry – here‘s a quick refresher – but the accuracy of the results depends on several factors.

Accuracy of the intrinsics

The supplied Intel® RealSense™ Depth Module D400 Series Custom Calibration program uses the traditional method, displaying a chequerboard to the camera in various poses and solving for the intrinsics. There are several issues with this methodology:

  1. This method establishes the intrinsics solely for the colour camera.
  2. Although it resolves to sub-pixel accuracy, it does so on a single frame, which is imprecise. The results of 3D calculations are extremely sensitive to errors in the field-of-view: A one-degree error in the vertical field of view translates into >11mm error at 1 metre. Concomitant errors in the horizontal field of view make matters worse and they are quadratic in the depth.
  3. The depth stream is synthesised by the stereo depth module and the vision processor. Imperfections anywhere in the chain (unforeseen distortion, varying refraction at different wavelengths, heuristics in the algorithms, depth filtering) may negatively affect the accuracy. One cannot assume that an apparently perfect colour image will produce ideal results in the depth map.

Non-linearity

This is readily observed with the supplied DepthQuality tool. When viewing a target at a measured distance of 1’000 from the glass, the instantaneous reported depth is out by ~22mm:

By averaging depth measurements over a period, errors in precision can be eliminated. Averaged over 1’000’000 measurements, my out-of-the-box D435 reports a range of 980.70mm – an error of 19.3mm. This is within the specified accuracy 2%=20mm but increases quadratically, as is to be expected. Fortunately, this non-linearity appears to be constant for a given camera and once determined, can be eliminated.

Focal Point

The focal point of the depth map is supplied in the Intel RealSense D400 Series Datasheet, for a D435 it is defined as being 3.2mm behind of the glass. Presumably due to the manufacturing tolerances of ±3%, the focal point may in reality be tens of mm away.

Mounting

No matter how precisely the camera is mounted, there will be errors between the mounting and the camera’s true central axis. Knowing them improves the accuracy when translating from the camera frame to the parent (vehicle or world) frame.

Solution

I have written a program that calibrates a camera based solely on measurements in the depth stream. It derives all the parameters discussed above by making several observations of a target with known dimensions. A much higher degree of accuracy is obtained by averaging over a large number of measurements. The optimal parameter values are then calculated, as a single problem, with a non-linear solver.

It is open-source, available on GitHub https://github.com/smirkingman/RealSense-Calibrator

Screenshot of an optimiser output:

Discussion

The comparisons presented above use the Z-range as the metric, as this is the metric in the reference documentation. The Z measure alone is only part of the answer, a more realistic metric is the 3D error of the point: the vector between the truth and the 3D point determined by the camera and software. Futhermore, just supplying a number doesn’t tell the whole story. Traditional error analysis supplies descriptive statistics, which give a value and a confidence known as the 68–95–99.7 rule, which allows us to make statements like “The error will be no more than Xmm 99.7% of the time” (3-sigma, or 3σ). 

The 3D error – the length of the vector between the true coordinates of the point and what the camera+software reported is:

The 3-sigma error is:

what this shows is that at 1.5 metres, the coordinates of the 3D point will be between 1’447 and 1’553 from the camera 99.7% of the time.

Nov 012018
 

A depth camera measures the distance from the camera to a point in 3D space. For a given point, the camera supplies the row and column on its ‘screen’ and the depth towards the point. It is worth pointing out here that classic depth cameras like the Kinect supply the length of the ray; RealSense cameras supply the range, or Z component.

Calculating the coordinates of the point is fairly straightforward trigonometry. Suppose a D435 camera is mounted 500mm off the ground, pointing at the horizon. 1000mm away there is an object 101.5mm high:

To warm up, the camera’s vertical field of view is 56°, so at 1’000mm half of the height is

    \[1000*tan(56/2)=531.7mm\]

The camera has 480 rows, so it will see the 101.5mm-high object at row

    \[((480-1)/2)+(-398.5)/531.7*((480-1)/2)=row\ 60\]

Bonus: It sees the object at an angle of

    \[Atan⁡(398.5/1000)=-21.7^\circ\]

Now we define constants for the Fx intrinsics, the centre row and the height of a pixel:

    \[VFov2=VFov/2\]

    \[VSize=Tan(VFov2)*2\]

    \[VCentre=(Rows-1)/2\]

    \[VPixel=VSize/(Rows-1)\]

    \[VRatio=(VCentre-row)*VPixel\]

    \[Y=Range*-VRatio\]

Notice the ‘Rows-1’ because there are 479 intervals between 480 pixels: row 239 points just under the horizon and row 240 points just above the horizon.

Then, for the example above we define our constants:

    \[VFov2=56/2=28\]

    \[VSize=Tan(28)*2=1.0634\]

    \[VCentre=(480-1)/2=239.5\]

    \[VPixel=1.0634/(480-1)=0.00222\]

and calculate the Y coordinate:

    \[VRatio=(239.5-60)*0.00222=0.3985\]

    \[Y=1000*-VRatio=-398.5 mm\]

The calculations for the X-coordinate are identical, replacing ‘Vertical’ with ‘Horizontal’ and Z is simply the supplied range.

Jun 122018
 

A quick review of the D435 with pictures of real-life situations; I wish I’d had this when I was considering buying a D435.

Unboxing is a breeze. The D435 is small enough to hide in your hand, in a sleek aluminium case weighing in at 71.8g. It comes with a 1m USB3 cable and a cute, flimsy little plastic tripod. Amusingly, the 28.6g tripod isn’t up to the task of supporting the camera once the 41.2g cable is plugged in and it falls over; I’ve given it to my grand-daughter for her Lego. The standard camera thread mounting underneath works perfectly for a real tripod; for an aligned mounting there are 2 M3 tapped holes on the back, 45mm apart but they are only 2.5mm deep.

All the software is available on GitHub, quick-start with the Intel.RealSense.Viewer.exe to get this, depth at 1280×720:

The toy tripod is in the foreground. Notice that I switched on the hole-filling filter, which is off by default. Switching to 3D with depth colours and quads:

and using the camera colours:

The red light meter to the right of the tripod reads 28 lux: I was astonished at the depthmap accuracy in such poor light.

On of the major failings of depth cameras is reflections, so my next tests were with glass. Here, an interior, again the light meter in the foreground reads 15 lux:

The right-hand side of the table is glass and thus appears further away, which is normal. The wine glass next to the light meter is captured perfectly. The model on the front-left of the table is a tiny Sterling engine. The Viewer has a nice zoom function:

The stereo matching doesn’t get the inner details of the wheel, but the outline is well delimited.

Next, looking outside from behind a window of 6mm + 15mm glass at 260 lux. My cursor was on the middle of the small hedge to the right of the wall, the 32 metres measured through 2 layers of glass in the depth stream seem quite reasonable:

Stepping outside to 1’800 lux (and light rain), there are a few artefacts due to the reflection in the water in front of the chair and the window on the right appears to be at the same distance as the trees, which makes perfect sense as that is what is reflected. Notice that the depth images cover a greater area than the colour images. This scene is extremely unfavourable for a depth camera: poor light, reflections, textureless walls and rain:

Zooming in on the flower pot in the foreground, it isn’t immediately clear if the flowers themselves are distinguished:

but a little image enhancement shows that they indeed are:

Panning left, there’s a buddha statue about 8 metres away. I positioned the cursor on the roof of the building above the Catalpa tree; the reading was 65.54 metres, which seems right (and is quite astonishing):

The buddha is clearly rendered in the zoomed depthmap, with only a handfull of dead pixels:

The camera button on the depthmap outputs the PNG, the RAW and the metadata:

Frame Info:
Type,Depth
Format,Z16
Frame Number,64305
Timestamp (ms),1528815686864.40
Resolution x,1280
Resolution y,720
Bytes per pixel,2

Intrinsic:,
Fx,639.315613
Fy,639.315613
PPx,637.479370
PPy,362.691040
Distorsion,Brown Conrady

which is a nice touch.

The SDK, librealsense-2-12.0 has the CPP sources for everything and wrappers for C#, Unity, OpenCV, PCL, Python, NodeJS and LabView. I tried the C# example SLN; after adding the reference to Intel.Realsense.DLL, they ran on first compile. The Depth tutorial generates a cute 70’s-style image made with characters (it’s my hand):

and the 2nd example – 100 lines of code –  with depth and colour:

Conclusion

Over the years I have studied many depth cameras, particularly for outdoor use: Kinect, Stereolabs, SwissRanger, PrimeSense, Bumblebee, to name but a few. None were satisfactory, either because they were blinded by sunlight or cripplingly expensive. The D435 works perfectly both inside (with IR) and outside (with stereo matching); it is cheap, resilient and accurate. I think it is going to be a revolutionary game-changer in computer vision.