Frequently Asked

• The all-important the bounding box
• Controls need to be placed precisely
• You need to use both displays in the feature editing window
• Verification is just as important as annotation
• How to handle obstructions
• Learn the keyboard controls for greater precision

Getting Started

• It will take some practice to learn to recognize the subtle variations in shadow that show you where the edges of some features are (the nose is particularly difficult). It may help to stand in front of a mirror and use your fingers to feel for the boundaries of various facial features on your own face, and then take your hands away to see what those boundaries look like. It will help of the lighting you use with the mirror is in front of and well above your head, as is the case in the MASKS data set.
• In defining what is or is not part of a given facial feature we are not trying to achieve some notion of objective truth. In particular, we are not looking to make choices that a human anatomy expert would agree with. It is far more important to simply be consistent about the choices you, as a group, make across the whole image set. During the early part of the annotation process, you should expect some disagreement on what criteria to use, but the hope is that we will come to a consensus relatively quickly that can be applied uniformly throughout the image set.
• Is pretty close good enough? Unfortunately, it's not. Although we are not looking for feature boundaries that plastic surgeons would agree with, that is not to say that precision is not required. On the contrary: VisageMap is designed to allow placing the bounding box and control points as precisely as the image allows. Placing a tight bounding box and positioning each control point at its best possible location are both critical to the success of the project. One or two pixels out of position absolutely does make a difference: if you can see the feature boundary, there should always be a best position for the bounding box and for each control point. Please keep this in mind both when you are acting as the primary annotator and when you are verifying someone else's annotation. You should review both the Feature Rules and the Editing Process sections of this document to help you understand how the tool can help you achieve the necessary precision.
• If you are having a hard time seeing the details in the images, it might help to adjust the brightness and/or contrast setting in your computer monitor.
• Check out the training sample files, which have been updated to include two fully complete annotations, one good, one with many mistakes.
• If anything in this document is unclear, please don't hesitate to bring it to my attention. If you need to schedule some one-on-one training time with me to improve your annotation skills, I'm happy to make myself available for that.

The Big Picture

• A correct bounding box is vital. All control points depend on where the initial bounding box is placed, so this is by far the most important thing to get right. The bounding box needs to be placed so that its right edge just touches the right-most point(s) of the feature you're working on, the bottom edge just touches the bottom-most point(s) of the feature, the left edge touches the left-most points, and the top touches the top-most points. The fit of the bounding box must be tight on the boundaries of the feature. Every single control point is generated based on the position of the bounding box, so if the box is not correct, you must delete everything and start over. Stated another way, if the bounding box is not placed correctly, no matter if the control points appear to be in the right place, none of them are valid without a correctly placed bounding box, and I cannot use them.

  You will almost certainly find it difficult to place the box correctly at first, and you will need to delete any control points and try again. The tool can help you spot bounding box problems in several ways:

  • Refer to both displays. The large display can show you when the box is too big, i.e. the edge of the feature does not touch the sides of the box. The small display can show you if the box is too small, i.e. there is part of the feature outside of the box. Both can show you if you have a correctly sized box that is out of alignment.
  • At any time when placing control points, you can turn the grid back on - the outer edges of the grid are exactly where you placed the bounding box.
  • If part of the facial feature is outside the box, and if you later try to move a control point outside of the box to its correct position on the edge of the feature, the point will get marked as invalid (its line will turn red and the status message will change at the top of the window). This means that the bounding box is too small or misaligned, so you will have to delete the control points and correct the problem.
  • You can turn on inspection mode at any time, and you will see what your verifier will see. As described below, if you turn on the grid in inspection mode, both displays will be shaded to show the boundaries of the bounding box. If you have a tight bounding box, some of the control points should have tips right on the edge of the box, or very close to it. It is an especially good idea to self-verify using these tools before committing your changes: we'll make much quicker progress through the data set if you catch your own bounding box problems.
• Each feature has its own control point quota. If the feature is required and the quota has not been met, it is red in the navigator. If it is finished, it is colored green. The current control point count and the quota are listed as part of the status information in the editor window. If the quota is zero, you need only place a bounding box. When you commit a bounding-box-only feature, the box will be shown in the main window in red rather than green - this is just to visually distinguish a bounding-box-only feature from one that has control points that happen to be arranged into a rectangle (this won't happen in faces obviously, but VisageMap is designed to be useful for other kinds of photo annotation work).
• Once you've fulfilled the control point quota, don't worry about it if the outline some of the facial feature is being missed in the spots where there is not yet a control point. It is only important that each control point, taken by itself, touches the edge of the facial feature. We can and will improve the fit by adding more control points later, during a separate round of annotation. See below for more on the control point editing process.
• Left-right symmetry is a very useful check on getting the control points placed correctly. You should never expect to find perfect symmetry in a facial feature - you will almost never find it, but the asymmetries are usually somewhat subtle. Try turning on the grid, and imagine folding it along the blue vertical axis. Compare the position of a control point with the one at the corresponding position on the opposite side of the face. If its position is significantly different than its twin, you should suspect a mistake. There are a few cases of genuine, substantial asymmetry in the image set, but please convince yourself that any large asymmetries are real before committing the segment.
• What if there is something in the way? If there is something in the way of the true boundary of the feature, you should make your best guess as to where the true boundary of the feature is. You should never follow the edge of the obstruction. This will most often arise in the case of eyeglasses hiding parts of the eyes or the nose, or the subject's hair hiding part of the hairline or other edges of the facial region. To make a good guess, one useful tool is symmetry, which the grid helps you determine. Another tool smoothness of the overall boundary: observe the locations of the the nearest control points where the boundary is visible, and place the guessed control point so that there is a smooth curve between the non-guessed points.

  Often there will be more evidence for you to go on: for instance, if the nose supports on eyeglasses hide edge of the nose, but are themselves clearly visible, you can use the positioning of the supports to make a very good guess on the nose boundaries. If the eyeglass frame hides part of the eye, you can use the curvature of the eyelids or the eyelashes to guess where the eye boundary must be on the hidden region.

• Please take verification tasks seriously. Catching errors and inconsistencies, large and small, is vital to the success of the project. Please check over each and every annotated feature carefully, cycling through every control point, paying attention to both displays. Turn on the grid to check that you agree with the placement of the bounding box, and that it agrees with the control points well. If the bounding box is properly placed, there should be control points that lie on top of the box or very close to it on every side. Starting with version 1.1.1 of VisageMap, when the grid is turned on in inspection mode, the edges of the bounding box are shown in both image viewers: if the bounding box is as tight as it should be, the point of at least one control point should lie on or extremely close to each side of the bounding box. If you do not agree with the other annotator's choices, do not hesitate to adjust or delete control points. Precision is incredibly important for this project. Since every single control point depends on where the bounding box was placed, if the bounding box was not placed well, you have no alternative but to delete all of the control points and start over. This is a necessary part of the process.

  You might also find it useful to zoom the main image viewer to 100% or more - this will allow you to see the feature boundaries with out any controls or shading in the way. You can toggle the visibility of the feature off and on using the Navigator window, allowing you to see the feature with no markings at all.

  If two annotators are making changes to only a few features, and are sending them back and forth for re-verification, it is fine to only pay attention to the parts that the other annotator has indicated (s)he has changed.

• Make use of the comments tab in each feature, plus the submission comments that you are prompted for at the end. This is how you communicate with your fellow annotators, and the more you communicate, the less likely you are to duplicate or otherwise interfere with each others work. In particular, definitely add comments if:
  • You feel it is necessary to add extra control points for a feature.
  • You are forced to guess where a control point should go. This can happen if you cannot see the edges of the feature due to the exposure being to dark or something is in the way.
  • You believe a feature is genuinely radically asymmetric.
• Every version of an annotation file that gets submitted to the MASKS server is kept, so if during verification another annotator makes changes to your work, your original work is still there. Starting with version 1.2.0 of VisageMap, there is a new entry in the server request menu in the Annotator Tasks window: Get Review Copy. Submitting this request will allow you to review any version of any annotation file that is stored in the MASKS server.

  Requesting a review copy will pop up a window for you to enter a subject ID number (which always consists of 4 digits) and a revision number. By default, you get the most current revision, but you may also type in any revision number. In particular, if you have already been assigned a (re-)verification task, you should have a commit log window for that subject. The revision numbers are the first thing in each entry in the commit log, and are marked with a letter r. You can choose any prior version mentioned in the commit log for entry into the revision number field (leaving off that leading r). You can then visually compare the differences in the usual way. These review copies are not tied to any task and can be changes freely, but cannot be submitted to the server. You can close a review copy by clicking the x on its tab at any time, and they will be automatically deleted when you quit VisageMap. The titles used in the tabs for the main image windows have been customized so that it should be clear which tab is which.

  When you are just starting out in producing annotation work, I will likely ask you to use the function to request some of your own work. This will allow me to give you more specific feedback on your efforts.

  One caveat with this feature: loading any annotation file into VisageMap is memory-intensive, so I have been forced to raise the memory limits built into VisageMap a little bit (from 256M to 320M). It should still run fine under the 384 megabyte minimum that I specified for the system requirements. This should be enough memory to have one assigned task file and one review copy open at the same time, but I would not recommend opening any more. If your computer has lots of memory (like at least 1 gigabyte), and you would like to be able to freely open as many files as you wish, I can tell you how to bump the memory limit even higher. If you are sticking to 2 open files at a time, and still run out of memory, I'd appreciate it if you let me know that is happening. Symptoms of running out of memory include getting notified of null pointer exceptions when trying to open an editor window, a broken image icon rather than a subject photo in the main window, and the memory usage toolbar will show that you are up against the 320M limit. If you think that you have run out of memory, you will need to quit and restart VisageMap as the tool will be in an unrecoverable state, and will behave more and more erratically. This is an unfortunate drawback to the way the Java virtual machine is designed.

Feature Rules

• An optional feature (shown in black text in the navigator), is optional in the sense that not every face has that feature. If it is really present in the face you are working on, I would like you to annotate it. The tool will not, however, prevent you from submitting an annotation that is missing optional features.
• When a feature's name and description refers to left or right for example Left eye, this is always the subject's left or right, not yours. Feature descriptions will try to keep this distinction clear.
• It is nearly impossible to reliably identify the true top of the nose, the nasion, using only the frontal photographs that we are annotating here. Instead, you should make the top of the bounding box for the nose coincide with an imaginary line connecting the bottom-center of each eyebrow (if one eyebrow is lower than the other, use the lower one). By bottom-center, I mean the highest point on the bottom edge of the eyebrow.

  It is easiest to get this right by making the box wide enough to touch both eyebrows at first, adjusting its vertical size and position so that it hits the bottom center of each eyebrow and the lowest point of the nose. Then adjust the horizontal size and position to match the actual boundaries of the nose. In many faces, once this adjustment is complete, the box won't touch the eyebrows anymore once you've adjusted the width, but as long as you are careful not to change the height, it will still be lined up right.

• Eyeglasses can sometimes make it very difficult to reliably annotate the eye areas. The frames can sometime hide large parts of the face, and sometimes the lens prescription is strong enough to significantly distort the eye image. In retrospect, we should have asked people to remove them, but we have what we have, so just do your best. Do add something to the comments section if you think the combination of poor exposure and glasses are leading to completely unreliable results.

The Editing Process

• There are some cases where the keyboard controls are superior to the mouse. Unless you have a good mouse and very steady hands, fine adjustments are likely to be easier with the keyboard. Also, when adjusting a bounding box bigger than the window, the mouse controls are all but unusable. I suggest adjusting one dimension at a time, e.g. first the vertical. Use the up and down arrow with and without the option key to make the box taller or narrower, and nudge it up and down. Press the spacebar twice in rapid succession to switch between views of the top and bottom of the box. Once you're satisfied, do the same with the right and left edges using the right and left arrow keys.
• Each time you place a control point, your task is to move it so its tip just touches an edge of the current feature. Move each control point one at a time, ignoring where the other control points are - the positioning of the current point should be treated as if it were the only control point. Don't worry about the overall fit to the feature, and do not worry about the edges that connect adjacent control points.
• Both editor displays are important. While adjusting each control point, you will need to keep keep track of what is happening in both of them. We provide both displays since otherwise, no matter how we show the annotation boundaries, the controls will be in the way of what you need to see. The two displays together allow you to see the entire unmodified image.

  Let's assume that we are annotating an eye for the rest of this explanation. First consider the main display (the larger one with all of the controls). The main display shades everything outside of the area contained by the control points green, thus this display is best for seeing spots that are not part of the eye, and you need to adjust the control points to exclude the non-eye parts. The smaller display shades everything in the opposite way: everything inside of the control point area is green, and everything outside is not. The small display, then is good for showing you parts of the image that are part of the eye, but are currently excluded, so you need to adjust the control points so that all of the eye parts are included in the green area.

  To make the required adjustments, in the main display, look at the constraint line (i.e. the line that shows where you are allowed to slide that control point). Focus on the points that lie on the contraint line between the center of the feature and the control point tip. You want to adjust the control point so that every point on the line that is part of the eye, between the center and the control point tip, is not shaded green. Everything that is not part of the eye needs to be excluded, and thus should be green. If you see a little bit near the control point that is not part of the eye, you want to nudge the control point to put the non-eye parts into the green area.

  At the same time, in the smaller display, you want exactly the opposite to be true: the control point needs to be adjusted so that every part of the eye between the point of the red marker and the center of the eye is shaded green, and everything that is not part of the eye should not have the green shading. If you see a little bit of the eye near the red marker that is not green, you want to nudge the control point so that all of it is green.

  Tip: Especially when the facial feature is narrow, such as the eye openings or the lips, the various controls can make the display more than a little cluttered. One way to reduce the clutter so that you can see control positions more clearly is de-select the selected control point. To do this, double-click anywhere in the main viewer that is not a control (the green area is an easy target). Click on any desired control point triangle to re-select a point for further editing. Another way to accomplish essentially the same thing is to simply toggle inspection mode on temporarily.

• Once you've moved the point close to where you think it should be, you only need to concentrate your attention on those few pixels closest to the control point. To get the very best placement, it is often helpful to use the keyboard to experimentally bump the control one or two notches in either direction, deciding which position is best. (By the way, if you use the keyboard to cycle through the control points, the smaller display will automatically recenter on the currently selected control point. If you use the mouse, the small display will center itself on wherever the mouse cursor is pointing in the main display.)
• It is possible that the control point's axis touches the feature in more than one place. In this case, move it to the edge of the feature that is furthest from the center. This rule will get more of the feature inside the segment boundaries. It is also possible that the axis touches no point of the feature boundary. In this case, drag the point out so that it touches the edge of the bounding box. The easiest way to do this is to drag it away from the center until the axis turns red, then use the keyboard to adjust it back toward the center just until the axis is no longer red.

  Of course, you should never move a control point so that it crosses the opposite side of the segment.

• Watch out for invalid control points (there is a status display at the top of the editing window that monitors their status for you). Control points that are invalid have a red axis, so cycle through them to find the culprits. Don't add another group of control points until you've made all of the current set valid.

Problems

• If you feel strongly that you need an extra control point group to do a feature justice, you can do that, but the tool will warn you that you are doing so. Please also add a brief comment for the feature explaining what this difficulty was.
• If you see an usual feature that is not in the following list, please suggest it to me. Some of these will be added during later rounds of the annotion work:
  • Non-optional features: chin, cheekbones, irises, philtrum (the depression between the nose and the mouth), ears
  • Optional features: scars, tattoos, wounds, freckles, age spots, facial piercings, prominent facial lines or wrinkles, chin cleft, cheek dimples
• If you run into an annotation task that you feel you cannot complete, you can use the Submit Stuck server request. This is similar to the Submit Finished request, but you must provide a reason for using it, and it will be brought to my attention immediately.
• If the tool crashes or reports an error condition, you may wish to locate the debugging log. I will very likely ask you to email it to me (and may be interesting to look through if you're curious what the tool is doing behind the scenes).

Miscellaneous

• There is no way to log out of the annotation server other than by quitting VisageMap - this is why the username/password/login button controls get disabled after a successful login.

Keyboard Controls