U.S. Burning Plasma Organization e-News
March 15, 2008 (Issue 18)

CONTENTS

     -- Director's Corner by Jim Van Dam
    -- Announcements
     -- Reports
         - ITER Featured Prominently at AAAS Meeting by Don Batchelor
     -- This Month's Feature Article
         - Validation in Fusion Research: Towards Guidelines and Best Practices
            by Paul Terry
     -- Upcoming 2008 Burning Plasma-Related Events

Dear Burning Plasma Aficionados:

This newsletter provides a short update on U.S. Burning Plasma Organization activities. Comments on articles in the newsletter may be sent to the editor (R. Nazikian rnazikian@pppl.gov) or assistant editor (Emily Hooks ehooks@mail.utexas.edu).

Thank you for your interest in Burning Plasma research in the U.S.!

Director's Corner by J. Van Dam

In the US, the month of March is traditionally a busy time for discussions of federal budgets for science funding (not to be confused with “March madness” associated with college basketball tournaments at this time of year!) One example is the annual Budget Planning Meeting of the DOE Office of Fusion Energy Sciences, normally held in mid-March. In relation to burning plasma science we comment below about this meeting and also about other recent developments and activities.

Budget Planning Meeting talks: This annual meeting of the DOE Office of Fusion Energy Sciences was held last week, March 11 and 12, in Gaithersburg, MD. My talk about the recent activities of the US Burning Plasma Organization is available online (http://burningplasma.org/reference.html). Also, Jeff Hoy (DOE-OFES) and Ned Sauthoff (USIPO) gave a combined presentation on the ITER Project and US ITER Project Office activities, and Stan Milora (ORNL) gave a talk about ITER-related enabling research and development activities of the Virtual Laboratory for Technology; both of these talks are posted at http://www.ofes.fusion.doe.gov/FY2010-BPM-Agenda3-7-08%20final.htm.

STAC Issues Working Groups: In November 2007 the ITER Organization had organized 13 working groups to address key issues with the ITER baseline design that had been identified at the STAC-2 meeting in September. A number of US scientists are vitally involved in this work as working group leaders and members. Rich Hawryluk, a member of the USBPO Council, who leads the international working group T04 on ELM control and who also is spearheading the overall US physics effort in all of these working groups, presented a talk describing the ITER design review physics issues at last week’s annual OFES Budget Planning Meeting. (His talk is available on the USBPO web site.) This week, all the working group leaders met meet on March 18 at Cadarache in order to brief the ITER Organization about their progress and determine what recommendations to present to the STAC at its April 7-9 meeting. These recommendations were reviewed the next day, March 19, by the ITER Organization and leaders of the seven ITER-partner Domestic Agencies.

Energy Policy Act Report: As reported in the January issue of eNews, the National Research Council had last year set up a Committee to Review the US ITER Science Participation Planning Process (CRISPPP). This special committee is charged with evaluating the EPAct Report that was written by the USBPO in 2006. Earlier this month the USBPO provided to the committee various statistics that quantitatively measure US participation in ITER. Some examples are the following: US contributions to the ITER Design Review (21% of total effort) and the STAC Issues working groups (23%); ITER-related experiments on the three major US facilities (≥50%); US lead- and co-authorship of papers in the ITER session at the 2006 IAEA Fusion Energy Conference (lead on 3 out of 12 orals, lead on 6 posters and co-authors on 6 others out of 44 total posters); US authorship (27%) for Progress in the ITER Physics Basis published last year. It is expected that the CRISPPP report will be completed by approximately the end of this month.

ITER session at 2008 Fusion Energy Conference: Every two years the International Atomic Energy Agency sponsors the Fusion Energy Conference. This year’s conference (http://www-pub.iaea.org/MTCD/Meetings/Announcements.asp?ConfID=165), the 22nd in the series, to be held October 13-18 in Geneva, will commemorate 50 years since the 2nd UN Conference on the Peaceful Uses of Atomic Energy, which was held in 1958 at United Nations headquarters in Geneva. The 1958 conference was a landmark because results from previously closed national programs on fusion energy research were declassified and the countries of the world joined together from that time in an international, open research effort to develop fusion for the benefit of all humanity. During this year’s conference, there will be a session of ITER-related papers—approximately 10 oral papers and 40-50 poster papers. The ITER Organization, in conjunction with the seven Domestic Agencies, will be responsible to recommend, select, and submit the papers for this session.

ITER postdoctoral fellowships: As reported on its web site, ITER recently signed a Partnership Agreement with the Principality of Monaco, under which Monaco will provide 5.5 million Euro over a ten-year period. As part of this agreement, five Postdoctoral Fellowships (400,000 Euro/yr) will be set up for young scientists to be trained over two years in research related to ITER. In addition, the agreement provides 150,000 Euro/year to establish an annual International Conference on ITER-related research to be held in Monaco. The Principality of Monaco will also host other scientific meetings related to the ITER project or the ITER Council. For more information, see http://www.iter.org/monaco-partnership/ and the announcement on the USBPO web site.

ITER and CERN: Last week, ITER also signed a Partnership Agreement with the European Organization for Nuclear Research (CERN). This agreement will facilitate cooperation in areas of technology (such as superconductors, magnets, cryogenics, control and data acquisition and complex civil engineering) and also in administrative areas (such as finance, purchasing, and human resources, including software programs). See http://www.iter.org/a/index_nav_6.htm for more details, including a photograph of Kaname Ikeda, the current ITER director-general, shaking hands with Robert Aymar, the CERN director-general who previously had been the ITER Project director-general.

USBPO leadership changes: At its recent meeting on March 5, the USBPO Council approved Steve Allen (LLNL) as the new leader for the Diagnostics topical group, replacing Rejean Boivin, who had served as its leader since the beginning of the USBPO in early 2006 when the topical groups were first formed. Steve had already been serving as co-deputy leader of this group with Jim Terry (MIT), who will continue in the position of deputy leader. Bill Heidbrink (UCI) recently stepped down as deputy leader for the Energetic Particles topical group when he became the secretary-treasurer of the APS Division of Plasma Physics. The Council is in the process of approving a replacement for this position. Finally, in order to bring the number of its members down to the level indicated in the USBPO Charter, the Council determined not to replace Steve Cowley (UCLA), one of its members, who is resigning in connection with becoming the next director of UKAEA Culham Laboratory. Our sincere thanks to Rejean, Bill, Steve, and Jim for their valuable service to the US burning plasma community.

USBPO annual report: A written report, covering the activities of the USBPO mostly during CY 2007, was submitted to the DOE Office of Fusion Energy Sciences. It is available on the USBPO web site.

Announcements

Positions at ITER
Very recently the ITER Organization posted an updated list of open positions for professional and technical staff personnel. The list is available at the ITER website (http://www.iter.org/a/index_jobs.htm) and also on the USBPO website.

Submit BPO-related announcements for next month’s eNews to Raffi Nazikian at rnazikian@pppl.gov.

Reports

ITER Featured Prominently at AAAS Meeting

Several thousand students, science and technology professionals, and media representatives had the opportunity to learn more about fusion energy and the ITER project when they visited the U.S. ITER exhibit at the annual meeting of the American Association for the Advancement of Science held February 14–18 in Boston. This year’s AAAS meeting, which focused on “Science and Technology from a Global Perspective,” included more than 150 symposia, plenary and topical lectures, and a variety of special events.

The exhibit featured displays on various aspects of the ITER project, a model and a 3-D video depicting the ITER device, a touch-screen fusion quiz, and several colorful plasma devices. Personnel from the USIPO, the ITER Organization, and Princeton Plasma Physics Laboratory staffed the exhibit, greeting visitors and distributing U.S. ITER brochures, fusion fact sheets, and one of the most popular attractions of the conference – “singing magnet” kits featuring the U.S. ITER logo.

The video, which was shown on a special 3D display, lasted 2 minutes and included music and professional narration. It contained an introduction with logo and map of participating countries, a plant view, a fly-through of the tokamak interior with a simulation of icrf wave heating near an icrf antenna, and a wrap-up comment showing the Earth from space and a galaxy. Movies of the tokamak fly-through and icrf simulation are available in various resolutions. A low-resolution version is posted on YouTube at http://www.youtube.com/watch?v=txdZSAbRnlM.

The main people who worked on the exhibit/video/quiz were: Jamie Payne, Bonnie Hebert, Cindy Lundy, David Williamson, Don Batchelor, Sean Ahern, Paul Fogarty, Fred Jaeger, Stan Milora, Dave Rasmussen, Phil Ryan, and Ron Sheldon. Others who contributed and/or served as exhibit hosts included Ned Sauthoff, Brad Nelson, Carl Strawbridge, and John DeLooper.

Figure 1: ITER Display at the 2008 AAAS Meeting

Feature

by Paul Terry

The Verification and Validation Task Group of the BPO was organized in January 2007 and charged with examining verification and validation as defined in other fields, and formulating procedures and practices applicable to fusion research, taking into consideration special needs and issues. The task group has submitted a technical report to the USBPO Research Committee. This report has been accepted for publication in Physics of Plasmas. A synopsis of the report follows:

Validation in Fusion Research: Towards Guidelines and Best Practices

A synopsis of a report by P.W. Terry, M. Greenwald, J.-N. Leboeuf, G.R. McKee, D.R. Mikkelsen, W.M. Nevins, D.E. Newman, and D.P. Stotler

Verification and Validation Task Group, US Burning Plasma Organization Full text available as an arXiv preprint: arXiv:0801.2787v1

The stated OFES goal of achieving predictive capability in magnetic confinement research will require computational models that have been shown to be valid under widely accepted standards. This can be achieved through the processes of verification and validation (V&V). Verification shows that a numerical algorithm correctly solves a mathematical model within tolerances. Validation shows that the mathematical model faithfully represents a stipulated physical process, again within tolerances. V&V have been formalized in other communities. The rigorous quantitative procedures and standards attached to these activities are not yet practiced in fusion research. Particular constraints and challenges in fusion make this difficult, including budget and manpower limitations; complexities of modeling that require integration of distinct, physically realizable models with vastly different scales, representations of plasma physics, and computation requirements; the occurrence of multiple equilibrium states, bifurcation dynamics, and extreme sensitivities; and limitations on measurement that make data on many quantities unavailable, and require a priori modeling to interpret measurements. These difficulties will require adaptation and evolutionary development, particularly of procedures for validation. To begin that process, this report focuses on validation as the less developed and prescribed part of V&V, and proposes procedures and best practices. It is assumed that verification, following the methodologies formulated by experts in computer sciences and elsewhere, with careful and accessible documentation, is a necessary companion to validation.

An examination of comparisons between experimental results and models in fusion sciences reveals a number of problems intrinsic to the process. These include the necessity of 1) confronting discrepancies between model and experiment in a way that leads to characterization, understanding and quantitative measures of the discrepancies and their sources; 2) finding methodologies for detecting fortuitous agreement and its ramifications, particularly in other quantities for which comparisons may not be possible; 3) identifying approaches to treating sensitivity, its effect on comparisons and interpretation of comparisons; 4) developing comparisons that differentiate between verified models with meaningful physical differences, while avoiding strong sensitivities that make agreement with experiment difficult.

The difficulties, constraints and challenges enumerated in the previous two paragraphs can be handled through a series of validation approaches that represent the core validation activities recommended in this study. These are qualification, quantitative error assessment, primacy hierarchy characterization, sensitivity analysis, and use of validation metrics. These approaches are briefly defined and discussed.

Qualification is a theoretical specification of the expected domain of applicability of a conceptual model and/or approximations made in its derivation. While a theoretical exercise, it is carried out in the context of the experimental plasma conditions under which a model will be applied. Therefore it may guide the design of a validation experiment or drive changes to a model. In qualification, the theoretical constraints of a model and the parameters of the experiment to which it will be applied are rated quantitatively as inputs to a composite validation metric. This may be straightforward if all experimental parameters lie comfortably within the domain of applicability. If parameters lie close to, or outside of the edge of applicability, nonlinearity in combining the quantitative contributions of neglected effects makes this a nontrivial issue.

It is useful to define errors as uncertainties or deficiencies. An uncertainty is a potential error in any phase or activity of the modeling process that is due to lack of knowledge. A deficiency is a recognizable error in any phase of modeling and simulation that is not due to lack of knowledge. Uncertainties arise in experiment from statistical error, uncertainties in equilibrium solvers and lack of precision in inputs, limitations in diagnostic sensitivity and resolution, deconvolution of line-integrated measurements, and modeling inherent in diagnostic signal interpretation. Uncertainties and deficiencies arise in modeling from equilibrium specification, resolution limitations, physics excluded by the model (kinetic effects, fast motions, boundary effects, etc.), artificial constraints (flux tube, fixed profile), and programming bugs. Errors must be quantified, and can be incorporated into validation metrics as independent confidence levels on measured quantities and model outputs as detailed in the extended version of this report.

The primacy hierarchy is a ranking of a measurable quantity in terms of the extent to which other effects integrate to set the value of the quantity. The primacy hierarchy allows tracking of the way uncertainties and deficiencies combine in measurements. For probe measurements of particle transport, primary quantities are density and potential fluctuation amplitudes, the E×B flow is a secondary quantity combining potential and a spatial scale, and the flux is tertiary, combining flow and density. The closeness of agreement between an experiment and a model is generally a function of primacy level. For example, comparisons of a gyrokinetic model and experiment show considerably better agreement for the level-3 flux than for the level-1 density fluctuations, suggesting that errors partially cancel in the higher level measurable, yielding better agreement. Measurements that tend to produce cancellation of errors may be incapable of providing meaningful differentiation between models with significantly different physics content. This tendency of error cancellation with rising primacy level is often seen, but errors may also add in certain measurements. It is important to make measurements and comparisons across the primacy hierarchy. Primacy level provides a method for combining different measurables in a composite validation metric.

Sensitivity analysis seeks to apportion the variation in the output of a model to different sources of variation. Sensitivities are intrinsic to bifurcation dynamics and profile stiffness. For example, modeled fluctuation levels and transport fluxes have a well-known sensitivity to profiles. Sensitivities have two general effects in validation. When strong, they amplify uncertainties and deficiencies, making small variations in inputs map to large variations in outcomes. This produces comparisons with large discrepancies. When sensitivities are weak the constriction of input uncertainties produces comparisons with small discrepancies. This can make it difficult to discriminate between models with significant differences. Good agreement in a non-sensitive quantity could simply be a reflection of low sensitivity rather than a demonstration of model validity. It is important to map out, characterize, and quantify the landscape of sensitivities in models being tested. Validation comparisons should be made over a range of quantities with different sensitivities. These comparisons can be rated using a composite validation metric.

Validation metrics yield a numerical rating of the extent to which a model is validated, taking account of quantitative information about approximations and validity regimes, uncertainties and deficiencies in both experiment and modeling, primacy hierarchy, and model sensitivity. Approximations, uncertainties and deficiencies can be assessed for comparisons of single measurements and incorporated into a simple validation metric. The results of a simple metric can be combined with ratings of primacy and sensitivity in a composite metric. There is no standardized or algorithmic procedure for quantifying individual assessments or combining them into a composite metric. These operations will have to be defined through a collective development process. A simple validation metric might consist of a numerical validation score and a meaningful standard by which to judge it. For example, the score could be the absolute value of the difference between an experimental mean and simulation data, averaged over the distribution of the latter. The standard could be a confidence interval extracted from multiple measurements made at each point of comparison. In composite metrics, simple validation scores computed at different primacy levels, and for quantities with different sensitivities are added with weights reflecting primacy level and sensitivity. A standard for assessing the composite score based on the weighting scheme and experience must also be developed.

Validation requires changes in the way fusion research is conducted. Specially designed validation experiments need to be conceived ab initio, and jointly by experimentalists, modelers, and theorists. The discharges associated with these campaigns will probably not showcase machine performance, but must be accorded runtime. Openness in reporting validation results, healthy skepticism about comparisons, and disclosure of difficulties and perceived failures need to be valued for their role in improving models. Blind and double blind comparisons would greatly enhance credibility. Validation activity needs to be appropriately rewarded within research organizations. Model analysts operating independently from model builders should be sought and cultivated. It is important to think about the design of new devices or utilization of existing specialized devices to remove complicating aspects of modeling and comparisons. A hierarchy of such devices and corresponding models would enable complicating factors to be unfolded systematically. Analysis techniques that unfold complexity should also be developed.

The procedures and approaches outlined above will require experimentation, development, and testing. Time will be needed for these refinements, and for the development of necessary infrastructure. To move toward these ends best practices are recommended. Qualification should be carried out and documented, with explicit specification of the region of validity. Verification should also be documented and include at the minimum convergence tests in time, space and particle number. The primacy hierarchy should be constructed, sensitivity analysis performed, and simple validation metrics calculated, with explanations for the physics behind agreements and disagreements. Composite metrics should be constructed and tested as means of honing and certifying predictive capability in the face of limitations intrinsic to plasma science.