Ensuring models provide reliable estimates and are fair, ethical, and unbiased
Image 1. ISAE 3000 Type 1 certification of out remote sensing-based biomass estimation models for smallholder farms
Our collective actions have created an excess of carbon in our atmosphere. One way to combat this is by implementing practices that involve planting biomass, such as trees, which can absorb the excess carbon. To achieve this goal, Space4Good in collaboration with Rabobank (Acorn program) is working towards enabling smallholder farmers transition to sustainable agroforestry practices and measure the carbon that is sequestrated in this process, thereby generating carbon credits.
"Carbon credits are a way to put a price on carbon emissions, making it more economically viable for companies to invest in clean energy and low-carbon technologies." The carbon credit market is rapidly growing and is expected to reach billions of dollars in the next few years, providing a new revenue stream for companies that invest in carbon reduction projects.
At Space4Good, the data scientists have been developing machine learning models for over two years to estimate biomass as a proxy for carbon sequestration mapping. However, before these models can be actually deployed, they need to be audited by an independent body. Finally last year in October, we were granted an ISAE3000 Type 1 certification, by Ernst and Young (EY), for all our internal controls and procedures associated with biomass modeling. In this manner, the ISAE300 Type 1 certification ensures that our biomass models provide reliable estimates for the subsequent generation of carbon credits.
Why an audit?
As a machine learning practitioner, there's nothing quite like the feeling of having your model successfully audited. After months of hard work and countless iterations, it's a validation of the effort put in and a confirmation that your model is ready for deployment. The process of getting a model audited can be a daunting task. It requires a deep understanding of not only the model itself, but also the underlying data and the potential biases that may be present. It also requires a commitment to transparency and accountability, as the auditing process is designed to ensure that the model is fair, ethical, and unbiased.
Pre-audit
Prior to the audit, data scientists at Space4Good spent countless hours analyzing the data to ensure that it was clean, accurate, and representative of the population that the model would be used on. They also ensured that the data was free of any biases that could skew the model's performance while implementing a number of measures to mitigate any potential biases in the model (using techniques such as data preprocessing, feature selection, and model regularization). These steps helped to ensure that the model was making predictions based on the underlying data and not on any extraneous factors.
The auditing process
Once the models were successfully tested internally, the audit process began with a thorough examination of the data. This was a rigorous process that involved testing the model on a variety of controls. The auditors also looked at the model's code and architecture to ensure that it was implemented correctly and that there were no errors. Finally, after weeks of testing and analysis, the audit was complete and our model was deemed ready for deployment. It is an incredible feeling of accomplishment, knowing that all the hard work had paid off. However , it is also a reminder that this is just the beginning. The model will continue to be monitored and evaluated to ensure that it remains fair, ethical, and unbiased.
Space4Good is proud of the hard work and dedication that went into the development and successful audit of our machine learning models. We are excited to see how the models will be used in the real world. We are now in the process of preparing for the Type 2 audit which will entail more in depth analysis on the how Spac4Good operates the controls related to the processing and modeling pipeline for estimation biomass over an extended period of time.
Would you like more information or are you interested in collaborating with Space4Good? Visit our website or contact us via hello@space4good.com.