FLL Judging Scoring Program Information
Program Requirements
Included Programs
Example Tournament
Navigating this Guide
Macro Security
USB Flash Drives
Team Names
No-Shows and Disqualifications
Additional Information
Toolbars and Menus
Score Sheet Generator Toolbar
Score Sheet Generator Menus
Judging Score Sheets Toolbar
Scorekeeper Toolbar and Menu
Advisor Toolbar
Advisor Menus
Technical Information
Process Guide
Program Requirements
The program requires one computer for each judging room if using individual room score sheets. When using the scorekeeper version of the score sheets, one computer is required for the scorekeeper.You will also need the following software:
Microsoft Windows (not tested on Mac OS X or with Mac versions of Office)
Microsoft Excel 2003
For Some Output:
Microsoft Word 2003
Microsoft PowerPoint 2003
Note: The program may work on earlier versions of Microsoft Office programs, for example Office XP/2002, or Office 2000. Some features may not work as expected, or at all. It will not work well (if at all) with versions of Office prior to Office 2000.
Included Programs
My suggestion is that you save a good copy of the downloaded ZIP file, and make a new folder containing all the software for each tournament you will be doing. Here is a screen shot of the folder that should result when you uncompress the scoring software. There are the following items:Output Templates – templates provided for use with the various forms of output. There are also some additional output template options available if you wish to use alternatives
Templates – templates for use with the scoring software for creating rubric score sheets for judges and scorekeepers. These templates are also used to generate team feedback.
Timer – a simple Java based timer. Source code is included. Thanks to Bonnie Schwartz, OH FLL Head Technical Judge!
_Latest Changes.txt – a text file that lists the latest changes included in this distribution
_READ_ME_FIRST.doc – a Microsoft Word document with some important information
Judge Advisor.xls – the main program for use to analyze and output judging results, and for awards deliberation
Score Sheet Generator.xls – the main program to use before your tournament to setup score sheets
Example Tournament
The example presented in this user’s guide has the following characteristics:36 Teams
Hybrid block schedule generated with the FLL Tournament Scheduler software, version 4.11.
Three judging rooms of each type, with two randomly named judges per room.
Practice robot performance rounds are included in the schedule. They are ignored by the judging software.
Randomlygenerated robot performance data. Thirty six standard deviations and means were generated, and then three normally distributed scores were generated using that information as input. Not tremendously rigorous, but it worked well without too much effort and generated scores that were not completely random. The robot performance scores were therefore at least a little more typical to what you might see in a real setting.
Randomly generated judging scores. Not completely random, but simply generated.
Navigating this Guide
I’ve developed this web guide using Microsoft Word on Windows with Internet Explorer v. 7, so I can’t guarantee how it will look on other systems. It isfairly simple, so it shouldn’t look too different on other systems.Most of this guide can be navigated like any other website. There’s no fancy navigation. Using your browser’s ‘Back’ button is usually a safe bet. There are a couple notes, however.
Most of the software is demonstrated using ‘screencasting’, which means you see in movie or screen shot fashion exactly what you will see when you operate the program. It’s like looking over the shoulder of someone who is experienced using the program. These screencasts are in Adobe Flash format. Typically in Internet Explorer, to activate and especially interact with a Flash animation, you have to click on the animation.
Periodically the animation will stop waiting for user input. At this time you will see a green arrow somewhere in the animation. Clicking on the arrow will continue the animation.
When you are completed, you can click your browser’s back button to return to the main contents of the guide.
Macro Security
These programs rely on macros, so you will likely get security warnings when you use them. Nothing I distribute has any viruses in it, and should not cause any problems. I am looking into get a certificate so I can digitally sign the macros to insure safety, but until then you will have to trust me. Please look here to review security settings.USB Flash Drives
Do not run the score sheets directly from a USB flash drive, especially if using autosave. You are likely to have file corruption or experience program slowdown if you do.Team Names
For the Score Sheet Generator to read judging schedules properly, all team names should be free of punctuation, especially commas. A team name with a comma in it will cause an error and will incorrectly read the schedule. To correct this just make sure that none of your team names contain commas.When using the Advisor program to read Robot Performance Scores, commas are acceptable in the file output from the National Instruments scoring program.
No-Shows and Disqualifications
You can account for a team that doesn’t show up for your event during the score sheet signing process. Judges will be asked during the signature process if a team doesn’t have all rubric items evaluated (if that check is toggled on) whether or not that team is a no-show.Disqualifications can be entered using the Advisor and Editing the Nominations.
Additional Information
The Advisor will identify the teams you “should” invite to a championship event. It picks the top XX% based on the standard algorithm: 25% Teamwork, 25% Robot Design, 25% Robot Performance, and 25% Project. Please verify that the teams the program picks are the same ones that you would choose. Do this before you give them invitations.Robot performance ranking will go to three rounds to break ties. There are no provisions for timed completion. It also assumes three rounds, no elimination rounds.
Auto-save is available on the judging score sheets. The file will be automatically saved every 10 minutes. This must be turned on by the judges.
If you accidentally click an incorrect button during sheet generation, answer cancel to any additional dialog boxes, and quit Excel without saving anything. Then start the process over. The error checking is rather minimal, so sometimes it’s best just to start over.
Toolbars and Menus
Program functions can be accessed in two ways, either through a toolbar near the top of the window, or via a menu item along the top of the window. The toolbar should be fairly easy to find, and the menu names all begin with FLL, so they shouldn’t be too hard to find either. If you need to know what toolbar buttons do, hover over one and a little balloon will tell you what the button does.Shown below are the toolbars and menus for the various programs.
Score Sheet Generator Toolbar
From left to right, the functions are:Return
Sample Questions
Erase Schedule
Read Schedule
Input/Edit Judges
Pit Mapping
Create Teamwork Sheets For Displayed Room
Create Robot Design Sheets For Displayed Room
Create Project Sheets For Displayed Room
Create All Sheets for All Rooms
Create Scorekeeper Score Sheet
Configuration
Reset Summary
Score Sheet Generator Menus
Judging Score Sheets Toolbar
From left to right, the functions are:Return
Toggle AutoSave
Sample Questions
Clear Nominations
Sort High to Love
Sort By Time
Enter Judges Notes
Sign Score Sheets
Clear Scores
Enter Team Comments
5, 7, 8, 10, 15 Timers
Scorekeeper Toolbar and Menu
From left to right, the functions are:Go To Control
Go To Teamwork Summary
Go To Robot Design Summary
Go To Project Summary
Reset Summary
Finalize Score
Advisor Toolbar
From left to right, the functions are:Return to Event Summary
Initialize Advisor
Read Robot Performance
Read Judges Scores (Individual Rooms)
Read Judges Scores (Scorekeeper)
Review Teamwork Scores
Review Robot Design Scores
Review Project Scores
Review Competition Summary
Deliberations
Edit Award Nominations
Batch Input Award Nominations
Lock Award Nominations
Sort Nominations By Rank
Sort Nominations By Votes
Determine Awards
Review Championship Invitations
Create Team Event Feedback
Edit Awards Script
Output Awards Slide Show
Output Excellent Certificates
Print Team Feedback
Print Tournament Summary
Advisor Menus
Technical Information
If you have any problems with the operation of the software, check the following:In the Visual Basic Editor from within Excel, verify that the following references are checked in Tools – References (in the Visual Basic Editor, which can be viewed from the Visual Basic toolbar):
Visual Basic for Applications
Microsoft Excel 11.0 Object Library
OLE Automation
Microsoft Office 11.0 Object Library
Microsoft Forms 2.0 Object Library
Microsoft Visual Basic for Applications Extensibility 5.3
They should be checked, but please verify.
Also verify the following is checked: Tools/Options/Security/Macro Security/Trusted Publishers/Trust Access to Visual Basic Project.
Process Guide
Start with:Score Sheet Generator.xls – Excel program to generate single judging session score sheets and summary pages. Uses 5 templates included in this archive.
Judge Advisor.xls – tournament judging summary and awards tool
Schedule.csv – schedule file in CSV (comma separated values) format
You can now import the schedule directly using the FLL Tournament Scheduler (current v. 4.11)
You can also still use the format as described in the Score Sheet Schedule Format Guide.xls file
Scores.sco – Robot Performance scores output from the National Instruments scoring program
Process for Judging Score Sheets – uses Score Sheet Generator.xls
- Open Score Sheet Generator.xls
- Go to the Usage tab if you are not already there.
- Click “Erase Schedule” if there is old schedule information in the file
- Input the name of your event
- Click “Configuration” to view/modify configuration options:
- Awards Scenario and Awards Descriptions
- Judging checks
- Score sheet protection
- Language
- Click “Read Schedule” to read in properly formatted schedule. There will be a status bar message in the lower left corner indicating processing of the schedule information. This is not instantaneous, but should take less than a minute or two. The status bar will say Ready when complete.
- Enter judge information using “Input/Edit Judges”
- Save the file if you haven’t done so already
- Go to Pit Mapping tab (the last tab in the workbook) by clicking on “Pit Mapping”
- If generating sheets the day of the event, add teams to the list as they arrive to assign FLL numbers and team names to corresponding pit numbers – OR – If generating sheets before the event, assign FLL numbers and team names to the pit numbers you wish. This is the preferred method. This step is automatic and not needed if you use a properly generated schedule from the FLL Tournament Scheduler software.
- Return to the Usage tab.
- There are three schedule display areas viewable on Usage tab, one each for Teamwork, Robot Design, and Project.
- Select a room for which you want to generate the appropriate schedule information by using drop down menus in the Room Name boxes
- Click the appropriate “Generate JudgingType Sheets for Displayed Room” button on the toolbar, or click “Create All Sheets for All Rooms”
- A dialog box will pop up, with a default filename of JudgingType_RoomName.xls depending on the type of judging score sheets you are creating and in what room they will be used
- Repeat until you have all the judging score sheets created – one file for each room/type of judging. For example, if you have two rooms each for Teamwork, Robot Design, and Project you will have a total of six judging score sheet files.
- Distribute judging score sheets to each room, and perform judging. Judges should transfer sheets to the hard drive of the computer they will use, rather than running from a USB drive.
- If there are late arrivals, there are provisions to add their FLL Numbers and Team Names via the Pit Mapping tab when judging is completed. For example, if the last team in a particular judging room arrives after the sheets have been generated, that team will still be listed as “Team 25, Stand-In 25” or something similar. Wait until after they have been judged, then go to the Pit Mapping tab to enter the FLL Number/Team Name information. Then you MUST MANUALLY RENAME THEIR SCORE SHEET TAB. In the above example, find a tab labeled 25 Teamwork Scores, and rename it to FLL # Teamwork Scores. You should only have to do this for the latest of arrivals, i.e. those teams that arrive after judging starts. If you enter the mapping information sooner, the links within the workbook will not work, and the score information for that team will not be accurate. This is why I suggest you setup everything before the day of the event if possible.
- Each judging team will be required to “sign” the sheets. Click “Sign Judges Sheets” to clean up the score sheets so that they may be input into the Judge Advisor software. This will also verify all the data checks that the judge advisor has configured previously.
- When judging is complete, gather all the score sheet Excel files and make any updates to the Pit Mapping tab and rename sheets as necessary for any rooms that had late arrivals
- Go through the score sheets room by room to verify completion and perform a sanity check.
- ALL SCORES SHOULD NOW BE FINAL! NO SCORE CHANGES SHOULD BE NECESSARY AFTER THIS TIME. Any additional judging information that is learned now should factor into the deliberations for awards.
- Open Judge Advisor.xls
- Go to the Event Summary tab
- Click on “Initialize Advisor”
- Read robot performance scores and judging results
- Most of the data entry for this program is done via existing files, with the exception of the Invitation Percentage, which can be increased or decreased or simply entered directly.
- The scores are input using the score file that is output from the NI scoring program
- The judging results are input using the files generated by each judging room. Note that they can be imported all at once. There are as of yet no checks if you read in the same file more than once, i.e. duplicate file input.
- On the Deliberations sheet (Click “Deliberations”):
- Make sure that all the teams that have been nominated are listed.
- If you want to toggle nominations for any team, you can click “Edit Award Nominations”.
- Lock nominations after you are happy with all the teams that have been nominated.
- Rank teams for each award using the third column. Then you can sort by ranking to get a fully ranked award sheet.
- Do award deliberations
- Choose award winners using drop down menus on the Award Winners sheet.
- Generate any forms of output that you wish.