Weaving an Open Source Social Fabric

The first objective is potentially the most ambitious and transformative, as it requires a shift in mindset from one of a proprietary and compartmentalized approach to civil infrastructure analysis, design and management, to a more transparent, participatory and communal approach that inherently archives the process for future generations. This can only be achieved through an effective virtual organization with appropriate policies, governance structure, incentives, and controls. There is evidence that the Civil Engineering community is ready for such a paradigm shift: (1) the community as a whole does not have a research and development (R&D) commons like other sectors, e.g., Semiconductor Research Corporation, to better interface research and practice; and (2) most civil engineers already possess a natural desire to become involved in projects where human lives are directly impacted, creating a viable market for citizen contributors. Still the OSD-CI virtual organization itself has challenging requirements that necessitate modification of existing open source models, based on research conducted in this project, to determine how best to incorporate distributed Citizen Engineers into a faithful design process that ensures extreme trustworthiness in technical tasks, while promoting participation through openness and incentives. Thus, OSD-CI will produce a new open source social model that incorporates distributed human resources, software, hardware and knowledge into the design of critical structures.


The ultimate goal of every discontinuous innovation that results in a paradigm shift is to make it “unacceptable” not to participate. This certainly is an audacious expectation; however, this grand challenge creates the backdrop upon which this virtual organization must be formed. It is thus important to understand how the Technology Adoption Life Cycle will serve as the medium through with this conceptual VO will ultimately be adopted by its target community and how OSD-CI will address its two-fold attraction problem: attracting projects/jobs for the crowd from end users and attracting and retaining the Citizen Engineers that form the crowd.

 

Balancing Trustworthiness and Openness

Many open source systems began with no controls to foster maximum openness, e.g., Wikipedia. Interestingly, its founders later regretted that structure and the lack of a priori roles for experts. For this reason, subsequent VOs, including that founded by the PIs, have instituted editorial oversight from the founding of their organizations. Considering the high levels of risk and liability involved in the proposed effort, we argue that the competing requirements of trustworthiness and openness may be best optimized through a tiered collaboratory. This concept acknowledges that there is a wide spectrum of tasks with diverse requirements that not all members of the crowd would be qualified to address. At the highest tiers, projects would be increasingly complex and risky, may require sophisticated analysis tools and substantial computational resources, and in turn offer higher levels of compensation, but as a result, greater selectivity within the crowd. As the task requirements and contributing crowd size varies, so too would the mechanisms for aggregation of their findings. This produces a governance hierarchy offering a spectrum of participation from crowd sourced to sole sourced.

 

Membership Profiles that Build Trust

One of the major advantages of the proposed tiered system is that it provides a mechanism for building trust through transparent credentials and rating systems for all its Citizen Engineers. It would be a significant challenge to determine appropriate predictors to pre-qualify trustworthy Citizen Engineers to work within the highest tiers of Figure 3. Thus the best way to build trust is to devise a system that creates, updates and maintains reliable Citizen Engineer profiles that include relevant credentials and professional data, as well as the membership rating of the Citizen Engineer, based on the accumulated trustworthy ratings received from past contributions within the VO using systems common to e-commerce, e.g., sellers’ reputations at e-Bay and reviewers’ ratings at Amazon. Variations on these concepts will be explored including the enhancement offered by weighting feedbacks according to the membership rating or “status” of the person providing the rating and considering higher order statistics in the rating data such as a consistency measure for a Citizen Engineer’s work history. These ratings will not only create incentives, but they will also provide end users with information that they can use to pre-qualify Citizen Engineers for their projects and a mechanism to evaluate and even aggregate their submissions. The result would be a form of tournament mobility as shown in Figure 3, where Citizen Engineers would start off “competing” at lower tiers and as their efforts are aggregated and positively evaluated they would become qualified to “compete” at higher levels. This is similar to how an open source developer may initially submit patches, then earn “commit privileges”, rise to “release manager”, and finally be invited to the project’s “board of directors”. The tournament results in progressive winnowing of the cohort at each stage, with “losers” being retained but their options or tiers of participation being narrowed and lowered. With an open source framework, a tournament mobility model provides a way to have both openness and trustworthiness, to “recruit” reliable Citizen Engineers for critical upper tier tasks, to increase organizational effectiveness by enabling the best inputs of the crowd to surface, and, as discussed more in a later section, to incentivize participation of both end users and Citizen Engineers.   

Aggregation Models that Maximize the Crowd

Interestingly, scientists and engineers currently use the crowd construct. Both scholarly research and state-of-the-art design depend on the efforts of predecessors and contemporaries, and the crowd, via peer review, citations, and standardization, decides whose work is best. Thus the engrained mindset within the community is that ideas are presented with full faith that the ones that survive are the ones that deserve to (Intellectual Darwinism). In much the same way, OSD-CI will maximize organizational effectiveness (1) by enabling independent Citizen Engineers to conduct analyses and develop ideas and solutions, (2) by allowing competing hypotheses/solutions/designs to survive as long as possible, and (3) by aggregating and delivering final results to end users through various mechanisms.

 

Aggregation of Crowd Contributions

Three parameters are crucial in the specification of aggregation models: (1) Who are the end users, and what resources (specifications, data, information, tools) do they provide to Citizen Engineers? (2) Who can be a Citizen Engineer and how can they submit designs or analyses?  (3) What are the evaluation mechanisms through which submissions are assessed and aggregated into a final outcome?  The values of these parameters will vary depending on the tier of the collaboratory. Before discussing the nuances of each tier, it is important here to emphasize that, as a transparent collaboratory, any member of the general public can join OSD-CI and view its projects and resources, though their membership rating would ultimately constrain their contributions to appropriately tiered projects.

At the lowest tiers, Citizen Engineers are used for detection, classification, and categorization exercises where end users  supply data (e.g., images). Qualifications to be a Citizen Engineer in this capacity are minimal, resulting in a potential large crowd whose input can be aggregated by a variety of Type 1 strategies, as summarized below:

Goal

Requirement

Aggregation Mechanism

Reliability

Consensus

Similar analyses receive similar weight

Anomaly Detection

Exhaustive List

Ranked databases of identified problems

Adjudicating Between Competing Analyses

Decision Support

Rating and/or voting systems that consider stature of contributing Citizen Engineer

Synthesis

Collective Verdict

Compilation with ability to comment on and even alter contributions from others

At a higher tier, tasks such as designing a specific component of the overall system  require some engineering training. Restrictions on who can submit designs are needed, but could not be so restrictive as to stifle the diversity within the crowd that is essential to the OSD-CI concept. Thus the membership rating necessary for a specific task is specified by the end user, with more complex and risky tasks requiring higher membership ratings resulting in a smaller crowd of experts. In this case, the aggregation of products supplied to the design gallery by Citizen Engineers could be one of several Type 2 schemes, ranging from no aggregation, allowing the end users themselves to evaluate all contributions, to presentation of only select qualifying designs to the end user, using pre-qualified judges or the crowd itself through a transparent comment/contribution mechanism. 

At the highest tier, the challenging task is to provide some high level consultancy that may in the limit lead to sole sourced work, such as the peer review of a completed project in which case a “final” design would be provided to Citizen Engineers who would inspect, analyze and critically evaluate the design, potentially using a variety of the computational resources within the cyber-collaboratory. Because very high levels of specialization are required at this tier, the size of the crowd is quite small and membership ratings quite selective. In this case, aggregation may not be required (sole source instance) or may involve very basic mechanisms of accumulating information from a small pool of qualified Citizen Engineers.