Here are my session notes for the I-4 2022 conference.
How to start? Facilitate your business in a quantitative cyber risk program
Risk managers tasked with integrating quantitative methods into their risk programs – or even those who are just curious about it – may wonder, where to start? Where can I get the mountain of data I need? What if my key stakeholders want to see risk communicated in color?
Participants will uncover common myths and misconceptions, learn how to start a program, and receive tips on integrating analytical rigor into risk culture. When it comes to quantitative risk, ripping off the band-aid is a recipe for failure. Focusing on small wins in the beginning, building support from within, and adopting a positive bedside attitude is key to long-term success.
It was a challenge to gather all the information I wanted to cover in 30 minutes. Sit me down with a few beers at a bar and I could talk risk all night. This blog post is an add-on to the conference, linking to the resources I’ve covered and providing additional details. This message matches the flow of the conversation so you can follow along.
The only takeaway from the conversation is: Just be better than you were yesterday.
If you are considering or are in the process of implementing quantitative risk modeling in your risk management program and need to pause or stop for any reason, such as lack of internal support, competing priorities, departure of your executive sponsor, It’s good. There are no risk police to come and yell at you for using a heat map.
We – the royal us – need to get out of the risk matrix. The Risk Matrix has been studied extensively by those who study this sort of thing: data and decision scientists, engineers, statisticians, and many others. It is not a credible and defensible decision-making tool. That said, the use of the risk matrix is an institutional problem. Solving the deep problems of perverse incentives and canonized “finger in the wind” methodologies in the field of information security is not your job. Just do your best with what you have. Add rigor to your program where you can and never stop learning.
The four stages of a Quant Risk program
I have four general steps or phases to help build a quantitative risk program:
Pre-quant: What to expect when you expect a quantitative risk program – you are considering quantitative risk and this is how you prepare for it.
Childhood: You have chosen a model and a methodology and you are ready for your first steps.
Adolescence: You have done several quantitative risk assessments and you are fully prepared to rage against the qualitative machine. Not so fast – don’t forget to bring everyone with you!
Adult: Your program is mature and you are making changes, improving it and adding rigor to it.
(I never made it past adulthood.)
You can follow these phases in your own program, of course modifying as you see fit until your program is completely quantity-based. Or use as many or as few as you want, scaling your program to suit your organization.
Step 1: What to expect while waiting for a Quant Risk program
During this phase, you establish the fundamentals, undergo training or self-study, and increase the rigor of your existing qualitative program.
Training – Self-training
Read, of course, a lot of reading.
First, a few books.
Even if you don’t plan on adopting Factor Analysis of Information Risk (FAIR), I think it’s worth reading some of the documentation to help you get started. Many aspects of risk measurement covered in FAIR adapt well to whatever risk model you end up adopting. Check out Open Group’s white papers on OpenFAIR, webinars and blogs from the FAIR Institute and RiskLens.
Blogs are also a great way to keep up to date on risk-related topics, often directly from practitioners. Here are my favourites:
Structured training and courses
Add rigor to the existing program
The development of risk scenarios is part of any formal risk management/assessment methodology. Some people skip this part or do it informally in their qualitative risk programs. You cannot take this shortcut with a qualitative risk; it is the first place where the risk analyst assesses the scope of the assessment and begins to identify where and how to take risk action.
If you haven’t already, incorporate a formal scenario building process into your qualitative risk program. Document each step. This will greatly facilitate the transition to quantitative risk.
Some frameworks that have risk scope components are:
Adopt a model
What model are you going to use? Most people use FAIR, but there are others.
Collect data sources
Start collecting data sources in your qualitative program. If someone rates the likelihood and scale of a data breach as “high”, where could you go for more information? Write down these sources, even if you’re not ready to start collecting data. Here are some places to start:
Lists of internal data sources: Audits, previous assessments, incident logs and reports, vulnerability scans, BCP reports
External data: Your ISAC, VERIS/DBIR, Cyentia reports, SEC filings, newsletters, regulatory agency fines and judgments
Subject Matter Experts: Experts in each area for which you have a risk scenario; people who inform about the frequency of events and their magnitude (often not the same)
Stage 2: Early Childhood
You have chosen a model and a methodology and you are ready for your first steps.
Perform a risk analysis on a management decision
Find someone with a burning question and perform a risk assessment, outside of your normal risk management process and outside of the risk register. The goal is to help that person make a decision. Some examples:
Get stakeholders used to numbers
Stage 3 – Adolescence
You have completed several quantitative risk assessments and are fully prepared to rage against the qualitative machine – but not so fast! Don’t forget to bring everyone!
Perform more decision-based risk assessments
In this step, perform several other risk analyzes based on the decision. See the list in Step 2 for some ideas. By this point, you have probably realized that quantitative cyber risk analysis is do not a subfield of cybersecurity. It is a subfield of decision science.
Create a forecast database
Record frequency and magnitude predictions for each risk assessment you perform. You will find that over time many ratings use the same data or at least can be used as a base rate. Building a library of forecasts will speed up assessments – the more you do, the faster they will be.
Watch your bedside manners
This is the simplest advice and the one that so few people do. It’s an unfortunate fact: the risk matrix is the de facto language of cyber/technology risk. It’s in CISSP and CRISC, it’s an acceptable methodology for passing organizational audits like SOX, SOC2 and FFIEC and it’s what is taught in university curricula. When moving from both organizations and people to quantitative models, be kind and remember that this is a long game.
Recognize the hard work people have put into existing qualitative risk programs
Focus on improving the rigor and fidelity of analyzes
Talk about what I can do for you: help you make decisions
Do not do that :
Disparage previous work and efforts on qualitative programs.
Quote Tony Cox in the break room, even if he’s right. “[Risk matrices are] worse than useless.”
Force everyone to consume data one way – your way
Stage 4: adult
At this stage, the quantitative risk program is mature and you are making changes, improving it, adding rigor and getting people to follow the risk journey. Work on converting the risk register (if you have one) and formal risk program to quantitative as a final big step.
Thanks to your work, risk management and the risk register is not a daunting task; something you show listeners once a year. It is used as an integral part of activity forecast, helping to guide strategic decisions. It’s used by everyone, from the board of directors to engineers and everyone in between.
Here are some references to help you in this transition:
Sustainable quantitative risk programs
The last and final tip is how to make your program sustainable. You want it to last.
Colors and adjectives are fine, but getting stakeholders to think in numbers will make your program last.
Widely reach all areas of business for SME inputs.
Integrate your program into all areas of business decision making.
The biggest challenge I’ve found isn’t data (or lack of data), which is the most common objection to cyber risk quantification. The biggest challenge is that everyone consumes data differently. Some people are perfectly comfortable with a 5-figure summary, break-loss curve, and histogram. Others will struggle to make decisions with colors or numbers. Biases, comfort with quantitative data, personal risk tolerance and organizational risk culture all play a role.
I commend the work of Edward Tufte in helping risk analysts overcome these barriers and present quantitative data in a way that is easily understood and aids decision-making.
I’m always interested in reviews. It helps me improve. Please let me know what you thought of this talk and/or blog post in the comments below. What would you like to see more? Less of?
*** This is a syndicated Security Bloggers Network blog from Blog – Tony Martin-Vegue written by Tony MartinVegue. Read the original post at: https://www.tonym-v.com/blog/easing-your-company-into-a-quantitative-cyber-risk-program-33es5