Complex Aerospace Systems Exchange

Complex Aerospace Systems Exchange

Planning Committee Lessons Learned & Best Practices

27 September 2012

Program Structure

·  D. Dress: I strongly recommend that panels be no longer than 2 to 2.5 hours long with short presentations and time for many questions.

·  S. Boller: Perhaps the sessions should be more focused on the issues most often encountered while performing the subject activities (planning, integration, test,..), as opposed to an explanation by speakers of their projects. For sessions where presentations are to be given, those that answer previously submitted questions seem the most valuable.

·  A. Arrington: Management Roundtable did a great job in setting the stage for the CASE program. Ideas from the RT carried into CASE (BEST PRACTICE).

·  A. Arrington: 4 hour sessions are two long for panels; limit to 2 hours.

·  A. Arrington: The break was a distraction within a 4-hour session.

·  A. Arrington: Focus the scope of the panels; we were at a high level for the first year; have more specific topics for the panelists to address.

·  A. Arrington: Skip panelist intros, at least introductions when it is each panelist’s turn to talk. This takes too much time and disrupts the flow of the session.

o  Options:

§  Allow the panelist to introduce themselves to provide the proper context to their talk.

§  Have a summary chart of the panelists listed on the screen for all to read (high level intro info) so that the audience can see this coming into the session.

§  Use the session 2-2 brochure. This was a standard page, tri-fold with a session summary, list of panelists (with photos!) and intros, and space for notes. (BEST PRACTICE).

·  A. Arrington: Have to have better time management. While we want the interaction with the audience, we also need to keep the pace of the session.

·  A. Arrington: Probably need to do a little better job on the session summary. Phil did a good job in session 2-1, and Mark did a brief one in 2-2 (but the session time management got away from them). Having the Session Chair (or a co-chair), not act as a moderator seems to allow time for collecting the main themes during the session. Need to develop best strategy for this.

·  A. Arrington: Probably need to encourage more interaction between the panelists during the session.

·  A. Arrington: Make sure that there is a co-chair (back-up) for each the General Chair, Track Chairs and Session Chairs.

·  B. Hoffstadt: Most sessions were more interactive and had more dialog than other conferences. That was very good. But I think that some sessions with panelists spent too much time hearing from each panelist first. In one case, with 5 panelists, it took 1.5 hrs to listen to the first two panelists and there was very little dialog. Then there was a break and it took another 1.5 hrs to hear from the final three panelists. Then there was about 20 minutes remaining for Q&A and open discussion. This was a four hour session with only 20 minutes of open Q&A. This was a move in a good direction away from the typical conference format, but I think it would be much better for the panelist presentations to be limited to no more than 20 minutes. 10-15 would be ideal.

·  B. Hoffstadt: If a single session is long enough to justify a break, I don’t think it’s important to make that break concurrent with breaks from sessions in other tracks. Mainly because there were retention issues in many of the sessions I attended, with steep drop offs in attendees after the break. Concurrent breaks for intermingling between tracks is still valuable, but doesn’t necessarily need to be for every break. In other words, if everyone is together for the morning session, the lunch, and one afternoon break, I don’t think they need to be together for every other break during the day.

·  B. Hoffstadt: I think having everyone together in a combined session for the first and last of each day is a good idea. That would encourage and maximize intermingling of people while also providing common “anchors” or frames of reference for everyone from day to day. I think it would also encourage everyone to attend these combined sessions (attending the full day) so they don’t miss out on the big sessions that everyone else would experience. Whether the last session of the day is a daily “recap” or a stand-alone fundamental or common-interest topic is worth consideration and discussion. On the last day, a recap or lessons learned session makes the most sense.

·  R. Kohl: Having the mid morning and mid afternoon breaks in a different bldg than the technical sessions seemed to reduce the size of audience in the post-break sessions.

·  R. Kohl: We may need to consider even more ‘non-conference, anti-conference’ approaches, methods, ‘tools’, take home products. I think that our final product was still more similar to than different from the standard conference model.

·  R. Kohl: I think that separating our desired audience into PM, Chief Engineers and then practitioners is perhaps too fine of a separation. I think that if we had two major categories, namely PM and technical, then we could/would address all 3 groups without needing to separate the Chief Engineer group from the other ‘practitioners’.

·  L. McGill: Including speaker’s positions in the program (in addition to their organization) was good to establish their context and probably also helped with the marketing (especially since we had a number of highly placed panel members).

·  L. McGill: Need to list all of the CASE participants (panel members, moderators, other speakers, session chairs, track chairs, etc.) in the program list of participants.

Program Topics

·  A. Arrington: I think the Opening and Closing sessions for CASE did their job in setting the stage and wrapping up the event.

o  For the opening for 2013, provide a short summary of 2012, with a list of main themes, then focus from there on which themes we will cover during CASE 2013.

·  R. Kohl: More ‘practitioner’ oriented, ‘how to do it and why do it’ kind of info seems warranted. We had some excellent ‘things to worry about’ presentations, but adding some ‘why bother’ and ‘consequences if you don’t’ info seems helpful in order to allow practitioners to convince their mgmt and leadership to adopt some new or innovative approach/method.

·  R. Kohl: Complexity surfaced many times, in various contexts, and it seems that maybe having a session (or more) about Complexity, how to recognize it and what to do about it. Paul C mentioned this item during the closing session.

·  R. Kohl: Off nominal behavior, rainy day scenarios and other fault handling issues surfaced many times and we may want to consider a session (or more) focused on’ What should my system do when bad things happen?’ and how addressing this question should surface in different parts of the lifecycle.

·  R. Kohl: Doing more V&V in the early part of the lifecycle (left side of the SE ‘V’). We may want to have a session (or more) focused on just what could/should the V&V/I&T community contribute to the left side of the ‘V’ activities. Some candidates could be ‘supporting or forcing requirement reviews from a tester’s perspective’.

o  During the closing session, someone (maybe Misty?) mentioned that we should have more overlap, intermingling and/or cross pollinating between the Dev and the Test tracks. Maybe having a half day session (or more) where the first half of the half day is the Dev perspective of some topic/issue and the second half of the half day is the Test perspective. For example (see 8 above), we could have a session about ‘what Dev would like to get from Test in the early part of the lifecycle’ followed by ‘what Test thinks it could contribute to the early part of the lifecycle’. And perhaps a grand finale of ‘here are some V&V/Test activities that could be done early/earlier in the lifecycle (e.g. left side of the ‘V’).

·  R. Kohl: The need to ‘justify’ adopting a new method. So maybe some more focus on the business case (aka cost/benefit tradeoff, pro’s and con’s, etc) of why mgmt/leadership should consider adopting a new method/technique/process?

·  L. McGill: Several of the failure briefings were not. A couple of the ‘failures’ were thinly veiled successes (i.e. we had an issue and we solved it – no actual failure). I know this is tough to get people to talk about and maybe we can’t get completely there, but as a minimum we should press for the speakers to be detailed about what they plan to discuss (not just the general topic) in advance, so that the session chair can help shape it, or at least know what they are getting. Especially with these more sensitive topics.

·  L. McGill: Generally, the approach of the Session Chairs working more closely with the speakers worked extremely well.

·  L. McGill: The previously stated improvement to have shorter sessions in 2013, will help us to work more relevant topics into the program

Communication/Marketing

·  D. Dress: I also liked the idea of the tri-fold handout that provides brief bios and other information. This would allow for more content and questions during the panels.

·  S. Boller: I liked the handouts at the start of the meeting with brief speaker/panelist bios, contact information, and relevance to the topics being discussed as a way of using the time spent on introductions for technical material instead. An alternative could be a single handout for an entire track or CASE conference that could include (in addition to the information listed above) the major topics to be discussed at each session and room for notes after each item.

·  R. Kohl: The name of our Conf, CASE, was not entirely clear as to the nature of the content. I think that we might want to consider more emphasis of the PM and SE content of CASE in the name/title of the conference

·  L. McGill: We were late in nailing down the program content. If we had been able to get the official completed earlier we could have incorporated it into the marketing and I think it would have improved our attendance. We had a great buzz going on the intent of the conference, with people asking for the details, and we ended up with a great program and great speakers, but I fear that it was just a little too late to build on that buzz and have time for people to get funding approved and commit their schedules to it. Recommend moving up our timeline for 2014 by just a few weeks. I think this is doable, as we have a better feel for how this will all come together the second time around.

Logistics

·  S. Boller: When capturing lessons learned and best practices shared by participants, the context needs to be recorded as well (I took numerous notes that conflict with each other, so without their context they are meaningless). I would strongly urge these sessions be recorded on digital video (the most accurate form of notes that could be obtained), and made available to the planning committee. The legal release allowing this could be part of the conference registration process.

·  L. McGill: I appreciate Scott’s point, but at least in some sessions (e.g. speaking on failures) we will not get the candid discussion that we are after if it is being recorded. Although we all have to be aware that we may be recorded in any open forum, it does change the dynamic when it’s obvious and planned for distribution. Still, for some sessions it should be considered.

·  L. McGill: Several sessions abandoned the stage for the panel members to encourage direct discussion with the attendees. This may be something we want to adopt across the board (except for any large plenary sessions where it might be hard to see the speakers without a platform)

Management Roundtable (A. Arrington)

·  Suggestion from Tom Irvine – Practitioners and Chief Engineers; build on Mike’s management RT.

·  Per our Friday 9/21 telecon, I think we were looking at workforce issues and perhaps a pair of related roundtables/workshops (one management and one young professionals who are transitioning from discipline to systems).

·  The BEST PRACTICE is to get a cross section of the session chairs, speakers and panelists involved with the roundtable so that the themes from the RT can be carried into the CASE sessions.

Subjects for CASE 2013 (A. Arrington)

·  Obvious starting points are use the Track subject areas from 2012 (CS Development, V&V/integration/test, and Project Management). Need to drill down into these for more refined topics.

·  Common theme was workforce retention and training.

o  University of Capetown ideas (from Sophia’s session).

·  Safety. Could be a track; at least a sessions. Aviation Safety (great tie to the Aviation forum; can bring in industry as well as NASA research).

·  Social/human element/affects on CS development.

o  Social Science Panel

o  Political Science

o  Communication research in larger organizations.

·  From Track 2:

o  Aging tools.

o  Aging workforce.

o  Maintaining skills between programs.

o  Proper use of tools.

o  Involve workforce early

o  Understand requirements; define the end; how do you know success?

o  Need to change testing approach.

Suggestions for CASE 2013 (B. Hoffstadt)

·  We should explicitly address communication & the “soft” skills and factors as they relate to complex systems. But I’m not sure how or how much.