Risk/Benefit Analysis: Quantitative Means of Evaluating Technological Projects
Student’s Name
Institutional Affiliation
Course
Instructor’s Name
Date
Risk/Benefit Analysis: Quantitative Means of Evaluating Technological Projects
Geoengineering, cyber technology, nanotechnology, genetics, and other pioneer technological fields share a common property; their effects and implications must be inferred before their implementation or marketing. Quantitative risk-benefit analysis (qRBA) is one of the techniques used to assess and compare hazards and gains associated with a project or product. In recent years, qRBA has been effectively used in the pharmaceutical industry, providing years of risk assessment before the approval, production and marketing of a drug. However, there has been scientific criticism that qRBA is not an effective risk-assessment tool for technological projects as it is for medicine. The little available scientific literature on qRBA highlights application and practice standards inconsistencies. The usefulness of any risk-benefit analysis is dependent on how well ramifications and underlying assumptions in a project are understood. While qRBA is highly applicable in medicine and pharmaceuticals, it tends to be incomplete and less useful in technological projects due to the minimal availability of data, lack of standardization and incompleteness of existing tools.
Gaps in qRBA for Technological Projects
The most common problem in technological projects for qRBA is the lack of sufficient data for analysis. Technological innovation revolves around the research and development of emerging fields, concepts or hypotheses (Bauer & Brown, 2014). Such undertakings do not have comprehensive data, yet the accuracy and effectiveness of qRBA are dependent on the quality and reliability of available data. The gap makes qRBA ineffective for technological projects in environments where rapid decision making is required (Bauer & Brown, 2014). The gap also lowers its utility in new research areas. A quantitative assessment focuses on measurable and realistic data, calculating the impacts particular risks carry. A lack of sufficient data implies an inability to measure impact, which negates the real-time utility of the tool. In another scenario, technological innovation means added difficulties in identifying the subject of the analysis, especially in projects with a high number of irrelevant variables (Guo et al. 2010). A new research field with lots of data makes it challenging to identify parameters for qRBA.
While a lack of data is an issue in the use of qRBA, the availability of huge data sets is also problematic. According to a series of Reactor Safety Studies in the early 90s by the U.S. Environmental Protection Agency, qRBA was associated with a significant degree of omitting risks due to technical gaps in assessing risks from multiple sources (Fischhoff, 2015). The study involved professionals from different sciences, including physicists, anthropologists and geologists. Each scientific field had different decisions to make, resulting in the collection and analysis of different sources of information. Multiple sources led to differences in data translation, resulting in judgment biases (Fischhoff, 2015). The study reveals that when it comes to large data sets characterized by multiple sources, there is an increased likelihood of overconfidence and anchoring. qRBA is limited because its risk ranking is dependent on the key terms defining data sets. As aforementioned, it is challenging to set parameters for huge data sets. The differences in parameters contribute to confirmation biases as individual researchers stick to the accuracy of their approaches.
qRBA is limited in its utility as each research area has to develop its unique qRBA tool. Class readings outline the story of Gryphon and colleagues, who for over a decade conducted critical studies in biosafety for the National Bio and Agro-defence Facility. As per the studies, the researchers discovered that existing tools are not well suited for contemporary research issues (Fischhoff, 2015). Critical research questions cannot be answered with existing qRBA tools. Due to differences in study fields and objectives, individual studies must create new quantitative approaches for treating data. The Gryphon experts suggest that each project with a new research paradigm develop its qRBA methodology (Fischhoff, 2015). The assertion is the application of the risk assessment method could be contextual. However, past psychological studies indicate that when individual studies design their parameters, values, and measures, the likelihood of confirmation bias increases (Fischhoff, 2015). Every other industry is investing in a different scientific field of research. The increased fragmentation of qRBA adoption raises questions concerning its accuracy and repeatability.
Standardization, which plays a critical role in determining the effectiveness of integrated risk management, is lacking in qRBA. (Reed & Lavezzari, 2016) lament that there is no standardized framework for guiding how to create qRBA tools. Businesses are becoming increasingly aware of the diversity of risks facing organizations. However, it becomes difficult to construct event trees or establish parameter values for assessing risk without standardised frameworks. qRBA is effective in medicine and pharmaceuticals because it is clear which parameters, theoretical models and features to employ per stage of the drug approval process. A ‘one size fits all’ model might not work for all types of risks in technological projects, but it will be feasible in guiding independent risk-benefit assessors (Reed & Lavezzari, 2016). Because organizations must tailor qRBA tools to suit their distinct risks, there is little iterative dialogue between managers and scholars to improve the success of qRBA modelling. As a result, methods remain not standardized. A good way to begin the process of standardization is by integrating qRBA in small, self-contained technological projects.
A lack of standardization makes qRBA unsuitable for informing decisions and policies in technological projects. In a self-reported survey involving twenty independent risk assessors, procedural gaps were cited as a common impediment to the adoption of qRBA (Smith et al. 2020). The study outlined poor internal governance structures regarding how qRBA supports decisions. qRBA approaches are not mandatory, which complicates the data collection process. Projects entail a certain degree of collaboration between departments to ensure each unit contributes information. The point of concern in information sharing is not the methodology or communication avenue but the completeness of the process (Smith et al. 2020). The lack of a standard process raises concerns about the completeness of existing qRBA tools. A lack of mandatory approaches means a high degree of inconsistency across the tech industry. Such a gap will result in the research community failing to agree on which methodologies to prioritize when informing policy and practice (Wang, 2021). It is easy for qRBA to calculate math, but the lack of standardization negates the acceptability of the method or findings.
One qRBA tool is insufficient to provide a comprehensive picture of potential risks and benefits. Each distinct qRBA approach offers a different summary and highlights different implications for a technological project. Guo et al. (2010) performed a meta-analysis of scientific literature to identify existing qRBA methodologies used in the pharmaceutical industry. The same theoretical models, parameters and features were applied to each tool to determine the amount of risk in a subjective drug assessment (Guo et al. 2010). Each tool delivered a different finding. The distance in outcomes in some tools was significant. The findings led to the authors recommending the application of more than one qRBA method to ensure a more objective and transparent risk assessment (Guo et al. 2010). The study shows that a single qRBA methodology cannot provide a detailed risk-benefit profile. A lack of comprehensiveness might cause an internal resistance to change in technological projects. A team that employs qRBA to guide the implementation process is likely to fall back to the same routine.
The incomprehensiveness of qRBA approaches requires high levels of human expertise and collaboration to offset. A common impediment to qRBA adoption in Guo et al.’s (2010) meta-analysis was varied human expertise. Even though scholars have increased their understanding of holistic risk management, many people still do not know about qRBA, its methods, roles and purposes. Existing tools and methodologies are incomplete because there is insufficient professional collaboration in sciences relevant to qRBA. The report has established existing methods define terms and parameters in accordance to interests they seek to serve. As a result, critics often cite the growing importance of judgment in improving analytical calculations (Fischhoff, 1015). Progress has been made in the last decade, but more needs to be done in human training and development. Inter-professional collaboration must also be enhanced to keep pace with scientific advances (Wang, 2021). Human training should seek to modify human behaviour to negate confirmation bias and elicit expert judgment when considering the robustness of qRBA conclusions.
Unlike in medical research, no single individual or team can take command of a qRBA assessment in technological projects, lowering its chances of success. Technological advancements occur across sciences, resulting in benefit-risk evaluations being considered a ‘team undertaking’ (Smith et al. 2020). Drug studies often involve a single field comprised of several professionals. Therefore, it is easy to select a head researcher. Responsibility in technological projects has to be shared across multiple functions and departments, with no single group claiming ownership (Smith et al. 2020; Wang, 2021). Cross-functionality stems from the fact that technological projects tend to be large and complex, necessitating fragmentation for easy management. The operational structure in technological ventures heightens the need for cross-department collaboration for a more comprehensive picture of the project or organization. A high number of teams, fragments or departments increases the difficulty in qRBA collaboration. More teams imply more differences in perspective, parameters, key terms and features.
Counter Argument/Rebuttal
qRBA is a conservative approach that tends to understate a project’s value, reducing the chances of a false positive. Traditional quantitative evaluation methods, such as the net present value (NPV), consider time and money as the main risk assessment measures (Mathews & Russell, 2020). Such methods assume that the project will be successful in its execution, thus overstating its value. Recent developments, such as the Monte-Carlo simulation, consider real options too complex and volatile for corporate projects. The recent methods employ periodic or phased risk-benefit assessments to enhance confidence in risk reduction (Mathews & Russell, 2020). A phase-by-phase approach addresses the tendency for increased project valuation. Moreover, it highlights that the risk assessment methodology is consistent with agile development principles, applied in innovative development. qRBA will not reject or adjust a project solely based on time and financial factors, making it useful for technological projects with alternative objectives.
Advancements in information communication technologies address the collaboration problem impeding the use of qRBA. Bellanti et al. (2015) discuss the revolutionary qRBA software, Monte-Carlo simulator, that comes with in-built user interfaces to enable inter-professional collaboration. The powerful tool contains pages, user profiles, and open project plans to allow different project managers to add uncertainty and risk to the overall assessment and subsequent project scheduling (Bellanti et al. 2015). The simulator quantifies results and offers easy-to-understand digital templates for sharing findings, ideas and arguments. The communication method allows different users to be verbal when management or units push for unrealistic project goals. Enhanced collaboration reduces the likelihood of confirmation bias and enhances human judgment concerning the reliability of findings (Guo et al. 2010). The problem is Monte-Carlo simulator, and other such innovative qRBA tools remain unpopular in current project management practice.
Several quantitative methods for risk benefit analysis in technological projects exist but it is unclear how the techniques differ. Necessary is the development of a framework for defining, organizing and selecting quantitative approaches as per type of project to increase their utility. For instance, Boers 3 by 3 table does not require statistical models, making it useful for organizing results of the same scale. The multi-criteria decision analysis (MCDA) enables systematic judgment by including trade-offs in the risk-benefit assessment, making it useful for projects that need to determine their impact to society (Mathews & Russell, 2020). Research into methodological differences in qRBA will facilitate their increased adoption. The choice of quantitative approach should be reliant on the goal of the project and the nature and availability of data. Further research into the relative advantages in the quantitative methods will make it more possible to provide specific recommendations on the type of approach per technological project.
Conclusion
Quantitative risk-benefit analysis (qRBA) is in its early conceptual and practical development years, negating its applicability in large-scale technological projects. The risk assessment approach boasts years of effective and reliable use in medical and pharmaceuticals. However, the complex, fragmented, and novel nature of technological advancements render qRBA unsuitable. Technological projects can have little data due to new scientific research topics or lots of data because of multiple sources. Each scenario presents a technical problem for qRBA based on the inability to decide parameters or the point of focus and an increased likelihood of anchoring and confirmation bias. Research in qRBA has yet to establish a standard framework for creating and assessing qRBA tools and methodologies. A lack of professional consensus and collaboration hampers standardization. Risk-benefit analysis must be a critical component of any project’s risk management plan. Further research and professional collaboration are required before qRBA becomes accepted at the conceptual and practical level in technological ventures.
References
Bauer, M. & Brown, A. (2014). Quantitative assessment of appropriate technology. Procedia Engineering, 78, 345-358.
Fischhoff, B. (2015). The realities of risk-cost benefit analysis. Science Magazine, 350(6260), 527-536.
Guo, J. J., Pandey, S., Doyle, J., Bian, B., Lis, Y., & Raisch, D. W. (2010). A review of quantitative risk-benefit methodologies for assessing drug safety and efficacy-report of the ISPOR risk-benefit management working group. Value in Health: The Journal of the International Society for Pharmaco-economics and Outcomes Research, 13(5), 657–666. https://doi.org/10.1111/j.1524-4733.2010.00725.x
Mathews, S. & Russell, P. (2020). Risk analytics for innovative projects. Research, Technology & Management, 63(2), 58-63. https://doi.org/10.1080/08956308.2020.1707012
Reed, S. & Lavezzari, G. (2016). International experiences in quantitative benefit-risk analysis to support regulatory decisions. Value in Health, 19(6), 727-729.
Smith, M., Till, J., Stefano, R. & Marsh, K. (2020). Quantitative benefit-risk assessment: State of the practice within the industry. Therapeutic Innovation & Regulatory Science, 55, 415-425. https://doi.org/10.1007/s43441-020-00230-3
Wang, W. (2021). Quantitative methodologies and process for safety monitoring and ongoing benefit-risk evaluation. CRC Press LLC.