Chapter 5 Policy
The Policy area’s intent is to support the people who develop and maintain research software, and to support research about research software, to ultimately increase the science & research impact of software. It does this by focusing on potential initiatives (actions, mandates, incentives, directives) decision makers can take to have a broad influence on the area, sometimes thought of as leverage points, including strategies and decisions to invest. Policy happens at multiple levels, including national, funding agency, institution, and research group.
The Policy area’s activities are divided into two parts, research and advocacy. Research is the collecting and analyzing of data, while advocacy is the dissemination of the research results in such a way that it changes practices.
5.1 Subgoals
The changes that the policy area seeks to make include:
In funding agencies, direct funding of software maintenance other software sustainability activities is a core part of the mission, e.g., at NSF, this includes all program officers across all directorates
In universities and academic fields, positions for people developing and maintaining research software are available, recognized, and rewarded
In publishing, support and recognition for software as a core part of science is the norm
In industry, sharing best practices, coordinating efforts, and contributing to open source software projects is the norm
Open source software is recognized as a key element of open science and reproducibility
Diversity of software project contributors is Increased
5.2 Area Structure
The policy area comprises:
Director for Policy, an URSSI co-PI, leads the overall Policy area, working with the Policy Advisory Board and Policy staff to define and execute policy research and advocacy activities.
Policy Data Science Team: performs research aimed at informing policy, but not about policy itself, for example, with regard to software usage metrics, empirical studies would be most effective. Includes funded staff, URSSI fellows, and community volunteers.
Assistant Director for Policy Advocacy, focuses more on planning advocacy that in doing it, by understanding different stakeholders, and building a roadmap for how to effectively advocate (within universities, foundations, industry, national labs, etc.)
Policy Advocacy Team, focused on policy development (such as use cases and examples of practices that are effective for sustainability), which ultimately would be used for presenting turnkey ways to support research software to institutions, possibly integrating and building upon the data science team’s findings. Includes funded staff, URSSI fellows, and community volunteers.
Policy advisory board (appointed volunteers) who provide advice on potential policy area and activities
5.3 Resources
The policy area will require resources that include funding for salaries, for events, and for travel.
Funding for personnel (e.g., buying out parts of contracts):
- 3 FTEs, divided across Assistant Director for Policy Advocacy, Policy Data Science Team (potentially a postdoc), Policy Advocacy Team
Additional team members, not funded by the policy area:
- Director for Policy, an URSSI co-PI, funded by URSSI at top level
- Policy Advisory Group (appointed volunteers)
- URSSI fellows, working in Policy Data Science Team and Policy Advocacy Team
- (TODO: Fellows program to be run by URSSI Community area; these policy fellows might be funded by policy area if needed)
- Other community volunteers, in Policy Data Science Team and Policy Advocacy Team
Funding for workshops: TODO: TBD
- Potentially including a policy conference / workshop(s) akin to Aspen Institute / CODATA - $370k, also could be part of an annual URSSI event
Funding for travel/speaking: TODO: TBD
If there is less budget available, we would cut back on workshop funding
If there is less budget available, we would:
Expand policy program to have a summer program for undergraduate and graduate students focused on software / science policy (e.g., an REU or something like a google summer of code for policy / law students)
Hire communication staff to do outreach and engagement around policy topics (would be dedicated expertise in policy as opposed to general outreach)
5.4 Methods and processes
Initial activities are planned with respect to the set of challenges that follows, but other challenges can also be introduced. The URSSI leadership group will regularly review suggestions of new challenges, and will determine a rough desired level of activity across the challenges.
The policy team will then develop and scope activities to match these desired levels, with review from the Policy Advisory Board, which will also be able to suggest adding or removing challenges back to the leadership group.
The policy team will also maintain this listing of active, planned, and potential activities and its mapping to the challenges. Activities will be split into those that, if successful, are likely to lead to significant changes in 1-2 years, 3-5 years, and longer-term, with an initial goal of applying 50% of resources to the 1-2 year activities, 40% to the 3-5 year activities, and 10% to the longer-term activities.
5.4.1 Challenges
Initially, the policy area has the following set of challenges to address:
Career paths (including titles and evaluation criteria for hiring and promotion) aren’t well established.
It’s hard to measure the impact of individuals, especially in activities that are inherently collaborative like software development.
The academic credit model disincentivizes individual contributions to public goods / infrastructure
We conflate quality and impact of software and need to disentangle them.
There is a lack of recognition of the value and importance of software.
There is a lack of funding opportunities and stable funding for maintenance of software that is important but doesn’t have a generic market, and “lumpy” project funding (projects that are competitively funded for fixed periods, often with gaps between funded project periods) means that maintenance/sustainability can’t be reliably folded into project costs.
The development and maintenance community is much less diverse than the overall US population.
5.4.2 Partners
The Policy area will work with other NSF SI2/CSSI Institutes & Centers of Excellence (e.g. SGCI, MolSSI, IRIS-HEP), the Research Software Alliance, the Software Sustainability Institute, the US-RSE Association, the (UK) RSE Society, The Academic Data Science Alliance, CaRCC, and any other projects and organizations that also seek to address overlapping challenges.
The Policy area needs to build relationships with the following groups and organizations, and to work closely with them, particularly in policy activities.
Institutional and research leaders in the academic and national laboratory community, such as VCRs, CTOs/CIOs, who can help URSSI penetrate various institutions and disciplines
Representatives from professional academic associations
Representatives from industry (including those already invested and sold on OSS, as well as those skeptically interested)
Representatives from Open Source communities, particularly those that are already effective and working at scale
People working in science policy, such as in AAAS and the National Academies
Representatives from organizations and companies serving OSS and RSE communities (e.g. NumFOCUS, Code for Science & Society, GitHub, Code Ocean)
Representatives from organizations that represent other research support roles (e.g., librarians, data stewards, RSEs), to work together to promote all such roles
Representatives from organizations that focus on diversity and inclusion in academia, to encourage them to include software-focused roles where possible
People from the European Commission regarding European Open Science Cloud (EOSC), etc.
Representatives from organizations like the Research Data Alliance and FORCE11
5.4.3 Planned Activity Pool
This is the initial list of activities that URSSI will maintain, and choose from. The specific activities that will be chosen to start depend on the funding level for URSSI. Most activities include an amount of effort that is needed to accomplish them. Note that some activities include raising additional funds for those activities - that fund raising is part of the activity. Other activities, in the next subsection, are tracked but are beyond the scope of URSSI, and are more likely activities that others might perform, potentially in coordination with URSSI.
High-level activities include:
Maintaining the list of potential activities and prioritizing them, including getting inputs from the overall community. These inputs will be collected and recorded by all members of the project in their interactions with the community, as well as by community members, via tagged GitHub issues. Community members will be able to see this list and react with a +1 to issues they think are important, and URSSI will use these reactions as an element of prioritization. (ongoing)
Tracking completed activities and their consequences (ongoing)
Collecting examples of good and bad policy, structures, and interventions from industry and from other disciplines (potential fellow project)
Developing an advocacy roadmap for how to effectively advocate (to universities, foundations, industry, national labs) (policy person & advisory committee, spread over 3 months)
<IP_ADDRESS> Policy Research Activities
These activities involve research, such as gathering and analyzing evidence and data.
Faculty career path policy - research actions:
- Study tenure guidance and policies, and find/document ways that software work is measured and recognized. Publish examples of successes. (2 months)
Staff career path policy - research actions:
Document existing (known successful/viable, known failures) career paths for individuals creating research software (2 months)
Examine and document Industry career paths vs lab career paths vs academic career paths (1 month)
Create and maintain a mailing list for those interested in career paths (mailing list to be supported by Community area) (ongoing, low level of effort)
Create a clearinghouse of job descriptions with criteria for performance assessment; distill out design patterns or different categories for different roles. This could also include documenting salary levels for different job descriptions, and connecting the job descriptions with then learning modules necessary for these jobs (3-4 months, plus ongoing maintenance)
Examine how salaries for RSEs compare with those of other researchers? (needs to follow clearinghouse, 1 month)
Measuring the impact of individuals policy - research actions:
- Identify factors (manual, such as evaluations, and automated, such as crawling repos) that are part of impact and publicize them. Some known factors to investigate include quantifying code contributions, code review, mentoring. Determine good practices (for formatting or housing the information on these factors) so those factors are discoverable and/or queryable. (6 months)
Incentivize contributions to public software policy - research actions:
Gather data/examples that show when contributions to public projects increase your citation count or regular metrics (1 month)
Gather examples of successful use of individual contributions to public goods/infrastructure to gain academic promotion (1 month over 3-6 months)
Use regular surveys to better understand why people do and don’t contribute; get data to understand the value of public goods to community (1 month over 3-6 months each year)
Determine a way to make it easy for scientific communities to peer-review public good software (or contributions to such software), with the idea that peer-review is considered a mark of quality (ongoing, not well-scoped)
Research to counter the idea that publishing in the small set of “highest impact” journals is key, and expose actual impact of work instead (work with scholcom community, use what they have done, maybe adapt their work for software - maybe 2 months to start)
Disentangle quality and impact of software policy - research actions:
- Create checklists/review guidelines for different levels of peer-review for software; can be tiered, could issue stars or use another rating system; leverage information already available from journals and other resources. (1 week)
Recognize the value and importance of software policy - research actions:
Find science/discovery cases where software was particularly fundamental, particularly digging into software that is not so generally well-known (1 month over 3 months)
Perform case studies in sustainable software value, e.g., NetCDF, HDF5, DS9 image viewer, GCM model(s) — how much money was invested in these, and how much maintains them? (3-month fellow project)
Quantify the money that has implicitly funded maintenance and sustainability, and has been saved or lost in the shift to open source (e.g., where have funds been spent on one-off software rather than maintaining existing open source software, and where has an investment in software maintenance saved funds from being spent on one-off software) (3-month fellow project)
Calculate bus factor for a bunch of key software in disciplines (partner with CHAOSS, or perhaps 3-month fellow project)
Document maintenance and funding for critical packages for several disciplines, e.g. IRAF in astronomy (3-month fellow project)
Funding opportunities for software maintenance policy - research actions:
Review the landscape of funding opportunities for software maintenance (and gather data about them) and provide a public summary, then keep the summary up-to-date. (jointly done with ReSA? Or CZI EOSS?) (1 month and some ongoing work)
Gather case studies of successful commercialization of open source projects and encourage the research community to understand and make use of them (2 months)
Review the scope of maintenance needs by various research disciplines. Determine the order of magnitude of the maintenance backlog for research software (with CHAOSS and others, unscoped)
Diversity and inclusion policy - research actions:
Survey US research software projects to examine diversity of leadership and contributors (3 months)
Study/survey contributors who make a single contribution and those who make ongoing contributions vs membership in underrepresented groups (3 months)
Contribute to DISCOVER event cookbook](https://discover-cookbook.numfocus.org), working with NumFOCUS’s DISC
Other items with CS&S and NumFOCUS’s DISC, for example, creating a cookbook aimed at projects
<IP_ADDRESS> Policy Advocacy Activities
These activities focus on advocacy. They generally depend on policy research or previously gathered evidence and data.
General advocacy actions (not mapped to specific goals):
Work with REU programs? – set up a network of software-focused REUs? Or focus on software aspects of more REUs? (provide training, guidance, language for proposals, …) (2 month to start + ongoing)
Play universities (and their administrations) against each other (as has been done by the SSI in the UK to create RSE groups), foster competition, by highlighting software work (including internal investments to sustain projects that the university sees as important and prestigious) and successes (incl funded projects that focus on software) in some universities. Rank universities by something and publicize? (maybe with US-RSE) – via newsletter, blogs, press releases (run by URSSI community area? Would accept info from community, e.g. Laura Noren’s and Brad Stenger’s Data Science Community Newsletter (also work with ReSA to bi-directionally share info when appropriate)
Work in citation and credit (get publishers to recognize and highlight software in research) (with FORCE11 and others) (minor support of other activities)
Promote software management plans and software sustainability plans as part of proposals (3 months over a year, bringing together community and volunteers, work with ReSA and RDA)
Develop a set of options for how institutions can support research software (e.g., RSE groups, RSE careers, tenure and promotion guidelines) and disseminate it (with US-RSE, starting after year 1)
Create and operate a professional award program (working with other established organizations) e.g., ESA URSSI software award. The URSSI software contribution to research awards: URSSI/ESA (e.g. John Chambers software award from stats association). $10k funding available from URSSI (eventually domain societies would be asked to partially fund the awards). URSSI would need to define the categories, criteria/heuristics, etc. for awards beforehand. And find people (either staff or community volunteers) to run awards processes (accept nominations, make decisions) (2 weeks/year starting in year 2)
Work with NumFOCUS to provide guidance to projects on how to work with (including obtaining funding from) industry (sharing material with incubator area)
Offer training for software development proposals (or point to others who do)
Develop and disseminate best practices for software development proposals (with NumFOCUS or the Capentries)
Training/materials could be asynchronous, or in-person with NSF program officers as well, if they were willing to do so
Licensing (in partnership with OSI, Creative Commons, others)
- Provide (pointers to) guidance on licenses and copyright for research software (little work, partner with OSI and point to existing best practices)
Faculty career path policy - advocacy actions:
Promote best practices in recognizing and measuring software work in hiring practices and tenure letters via sample guidance documents for committees, for faculty, and for letter-of-reference authors. (needs focused workshop and follow-through, ~3 months over a year)
Commission a NAS report (similar to the one on Open Source Software Policy Options for NASA Earth and Space Sciences) on career pathways/progression in academia (for scope see last item in next set of bullets)
Staff career path policy - advocacy actions:
Seed “chapters” of research software folks (perhaps called URSSI chapters?) at existing universities / societies / organizations; create handbook, tools, best practices to support local organizers; assign an URSSI “coordinator” / community manager. These chapters could: talk about training, do consulting for problems, host hacky hours, study groups, software days, come together into a larger conference, perhaps an URSSI-wide event. Overall, this helps / grows / establishes the community (and make connections that could help chapter members meet the right person for their next career moves).
- (TODO: done with URSSI Community area; Policy part (1 month) is creating some materials, e.g., guidance on how to set up a chapter, how to align it to local activities, and how to run it.)
Work to promote and support the establishment of RSE capabilities at universities around the US. Provide advice on how to talk to university administrators (and which ones) about this, etc. (1 month, jointly with US-RSE)
Provide examples of language to universities they can use in HR/job ads for RSE positions (1 month, jointly with US-RSE)
Commission a NAS report (similar to the one on Open Source Software Policy Options for NASA Earth and Space Sciences) on {career pathways/progression in academia, recognition of software, evaluation of software developers, inclusion of software in hiring and promotion} Would need to define, then raise funds from outside URSSI (~$1m), then work with NAS to define a charge and to contract for the study. (4 months over 2.5 years)
Measuring the impact of individuals policy - advocacy actions:
Create champions (e.g. librarians) to promote and educate individuals and projects of these good practices (working with Community area) TODO: ref best practices section in Communities
Help individuals understand how they can best promote themselves (e.g., claim software works on ORCID). TODO: ref best practices section in Communities
Incentivize contributions to public software policy - advocacy actions:
Work to persuade high-profile individuals to make public contributions to public projects
Share examples of successful use of individual contributions to public goods/infrastructure to gain academic promotion, produce templates, advocacy toolkits and examples to help others to make their case. Similarly, work to persuade people that contributing to projects will increase their products within their normally accepted reward system (e.g., get more collaborators and papers; increase opportunities to meet/work with new/old collaborators; will build social connections/network)
Show how you can participate in public goods / infrastructure projects (e.g., how to structure an issue, how to write your first PR) TODO: ref best practices section in Communities
Produce templates based on successful use of individual contributions to public goods/infrastructure to gain academic promotion, advocacy toolkits, and examples to help others to make their case (1 month)
Reframe contributions as first class research products / objects (i.e., explain how building the best software is itself science, as it’s both discovery and creation). In order to do this, amplify existing efforts to build a taxonomy of such contributions to make it clear what those contributions are, and encourage people to claim/talk about/take pride in these contributions; work with publishers to highlight these contributions and the people who make them. Also advocate (materials, webinars, ambassadors, etc.) within academic communities (deans, faculty, science societies, review panels, funders) that public software contributions are research (also could be done by cross-disciplinary respected groups, such as the national academies)
Work to revise funder policies to ensure reviewers prioritize grant proposals that reuse, build-upon and contribute back to maintenance of public infrastructure
Create high-profile equivalent of “highest impact” journal for software and data work - need to reject a lot of work and move it to “lower class” venues
Recognize the value and importance of software policy - advocacy actions:
Publicize science/discovery cases where software was particularly fundamental, focusing on demonstrating impact of software (1 week/year, ongoing)
Raise awareness that software needs to be maintained (low level, ongoing)
Funding opportunities for software maintenance policy - advocacy actions:
Encourage funding agencies to require software management/maintenance plans (SMPs). Use them to couple the idea of maintenance to the idea of development, to make it clear that just doing development without maintenance doesn’t work. Note that this leads to questions about when maintenance should be stopped, and it’s also unclear how long-term maintenance would be supported (since grants are by definition time-limited). (1-2 months/year ongoing, work with ReSA)
Advocate for funding agencies to provide a funding pool for short-term maintenance grants for existing projects (e.g. CZI’s EOSS program). (1-2 months/year)
Advocate for universities to support maintenance of software developed by their university as part of research impact (or possibly technology transfer). (2 months/year over multiple years)
Encourage companies to provide funds and channel these funds into maintaining research software projects, perhaps via a review process with reviewers from both academic software projects and the companies. This could be done jointly with NumFOCUS or a similar organization, similar to Tidelift (1-2 months/year)
Diversity and inclusion - advocacy actions:
Advertise DISCOVER event cookbook, working with NumFOCUS’s DISC
Other items with CS&S and NumFOCUS’s DISC, such as advertising the cookbook aimed at projects
Reach out to projects to encourage them to mentor Outreachy interns and those from other programs (e.g., Linux Foundation diversity programs, https://opensourcediversity.org/#programs), and to work with Google Summer of Code, maybe include some matchmaking. Consider URSSI as an organization that applies to a set of these programs on behalf of a set of software projects.
Build community mentoring program focused on diversity
Work with RLadies, Women in Data Science, etc.
<IP_ADDRESS> Unplanned Activity Pool
The activities listed here are those that would support URSSI but are beyond the scope of URSSI. They are listed here to provide ideas to others, and URSSI would potentially be willing to encourage and support them, but likely without resources.
Staff career path policy - research actions:
- Comparative study of research success at universities with core RSE groups vs those without. (potential research topic, not to be started by URSSI)
Disentangle quality and impact of software policy - research actions:
Define what software quality means.
Define what software impact means and how to measure it. Impact can be looked at in many ways: scientific, economic, societal, etc.
Define how you know what software will be impactful (signals of high impact)
Capture the difference between quality and impact in written resources (blog post, paper, etc.)
Conduct a Delphi study or multiple Delphi studies across or between disciplines to determine a consensus on key indicators of software impact and related questions, such as: Does improving the quality of software improve its impact?
Perform randomized experiments of software impact where the treatment is improved software practices
Perform a retrospective study of software quality as related to “impact” of that software.
Recognize the value and importance of software policy - research actions:
- Study whether higher-quality software produces higher-impact science or more reliable science results
General advocacy actions:
- Talk about problems with current software licenses/models and suggest alternatives
Incentivize contributions to public software policy - advocacy actions:
- Run a help desk once a week on-line for people who are running into difficulties in making contributions and need help
Funding opportunities for software maintenance policy - advocacy actions:
Provide a system to match open source maintenance needs and open source programmers (e.g., classified ads), to be funded by URSSI, or URSSI could support proposals for such maintenance work (both morally and by providing guidance to the proposer)
Provide fellowships for programmers to “do good stuff” related to maintenance
Attach money for maintenance to some of the to-be-created awards for good software development, jointly named and funded with other communities
5.5 Work with other parts of URSSI
Training – use to learn about problems, use training events to disseminate products and advocacy
Community – provide topics for fellows, work with on newsletter and dissemination, host champions, contribute to and disseminate best practices
Incubator – develop both general and specific guidance for projects
5.6 Metrics/Milestones
Amount of funding provided by US public and private funding agencies for research software development and maintenance
Number of US universities with RSE-like positions
Number of US university faculty who successfully use software work as an important part of their tenure packages
TODO: more needed


Setting this option will mean that successful or unstable builds will be tagged in Dimensions as a baseline. You will then have a Dimensions baseline representing the built source to refer to in the future.
The baseline that gets created has a name of
[PROJECTID]_[JENKINS_PROJECT]_[BUILDNO]by default, where
(Note: The resulting baseline name must comply with restrictions on Dimensions baseline names. For example, if the baseline name includes the Jenkins project name, then that Jenkins project name may not include the colon character).


---
layout: page
---
Experience
Physics Professor
- Community College of Denver 2021 to Present
- Calculus based general mechanics & electromagnetism
- Develop lessons for both in class and online teaching
- Record conceptual videos for supplemental lecture material to introduce and expand on concepts
Physics & Astronomy Professor
- CSU Northridge 2019 to 2021
- Algebra & Calculus based general mechanics
- Introductory Astronomy for general education
- Develop lessons that convey concepts in understandable and relatable methods
Programming Instructor
- Great Minds Robotics 2018 to 2020
- Private education for children aging from elementary to high school
- Applied use of NXT Lego Mindstorms to create tangible demonstrations of computer coding to improve absorption and understanding of concepts
- Developed a physics engine computer program to adapt to a teaching curriculum in order to teach students more about the underlying physics of games