The Buffalo Bills experimented with a "black box" as a minor tool for evaluating college football players several years ago.
The box was a machine designed to test players' quickness. Touch pads were placed several yards around the box in all directions, and each pad would light up, or blink. The player would have to move as quickly as possible to touch each pad that blinked.
"We quit giving that after we had a guy who had the highest score on the black box last two weeks in training camp," said Dwight Adams, the Bills' player personnel chief. "The guy who had the lowest score on the black box is still playing for us -- and very well."
The black box experiment was a small reaffirmation for the Bills of one of their guiding draft principles: In judging football talent, there is no substitute for how a player performs on the field.
"Let's don't make this thing an exact science," Adams said. "It is not. The bottom line is production and athletic ability. Those are the two things you need to grade: Is this person better than what you're playing with, or is he going to help your team? That's the question you have to ask yourself in scouting."
That question will be asked over and over at One Bills Drive this week, as the team gets down to the nitty-gritty in its draft preparation.
Starting last Monday and running through Friday -- the day before the draft -- the Bills' scouts and coaches will meet each day and review all of the draft prospects, one by one. They will come up with a final draft board of about 380 players, broken down by position as well as listed overall, 1 to 380.
The meetings are the culmination of a year's worth of work by the Bills' scouting staff, which has played a huge role in the team's success.
The Bills had the most home-grown starters in the NFL last season and are on a streak of nine straight seasons in which their top draft choice has proven to be a productive player.
They will put that streak on the line Saturday when they are scheduled to pick 26th in the first round.
One of the keys to that top-pick success is the Bills have not gambled on players whose potential far surpasses their performance on the field. The Bills don't like to take underachievers, no matter how physically gifted they might be.
However, the tricky thing about the scouting business is that the last two months before the draft are geared toward assessing players' pure athleticism, not their performance on the field.
The main event of the month of February for scouts is the National Scouting Combine workouts in Indianapolis. That's where the 300 or so players most likely to be drafted are given physical tests by doctors and participate in athletic drills, such as the 40-yard dash, the vertical jump and bench press.
Scouts spend the month of March visiting college campuses for more individual workouts.
It's easy to fall in love with a "workout warrior" or get down on a player whose time might be disappointing.
"You can be so nit-picky at this point you can talk yourself out of a draft choice," Adams said.
"The fact is you can find holes in everyone," said Bob Williams, who scouts the Southwest for the Bills. "You've got to see the athletic picture. You've got to stay with what you see and what you believe."
Nit-picking over minor flaws happens in the case of even the best players. Dan Marino, after all, dropped to the 27th overall pick in 1983 and was the sixth QB taken that year.
Phillips recalls the '82 draft, when Marcus Allen was the dominant college player of his class but had run a disappointing 4.6 time in the 40-yard dash.
"We had him clearly No. 1 and we had the 10th pick," said Phillips, who was coaching under his father, Bum, in New Orleans. "We watched him slip to fourth, fifth, sixth, and Bum said, 'We're going to get him!' Then the Raiders took him ninth."
Adams and the rest of the Bills' scouts think the combine is overrated as far as evaluating players' ability. What makes the combine important is that it's where most of the players take a physical. The Bills won't draft any player who hasn't had one.
"You go to the combine and watch a guy vertical jump in a pair of tennis shoes and shorts and a T-shirt, and run through bags and do these rudimentary drills," Adams said. "Now if you can come out and say, 'This is a football player,' then you're in the wrong business. If you're that good, you ought to be selling your wares to all the teams."
If those tests were a foolproof judge of players, then it would be a science, and anybody could be a scout.
"Speed is important, but you can get fooled by it," said David W. Smith, who scouts the West Coast for the Bills. "If it's a wide receiver who caught 80 balls and you have a decent time on him, but then one day he runs a slow 40, what difference does it make if he ran slow?
"Or you could have a guy who was a starter all year and caught 13 balls, and he goes out and runs a 4.35 in the 40. So what? You have to really be careful about the blemishes that might change your mind about what a guy does on the field."
Andre Reed ran a decent but unspectacular 4.54 in the 40 coming out of college. Bucky Brooks, a Bills second-round pick in '94 who never made it, ran 4.36.
"If speed was the key to being a receiver, we'd just go get us a bunch of track guys," Adams said. "But quickness is actually probably a more important factor than speed."
Any single test, Adams maintains, pales in comparison to a scout's innate ability to watch a game and answer the question: Can this guy play?
That's what the Bills' scouts do during the college season. They visit each of the colleges in their region of the country in the fall. They watch films of each draft-eligible player, talk to people on campus and watch practice. Then they give each player a grade assessing his ability to perform in the NFL. They have to see at least three current-year films to grade a player. Each of the team's eight area scouts is responsible for about 50 schools.
A grade of 5.0 means that player has a 50-50 chance to make a roster. A grade of 5.5 means that player should make it in the NFL, Adams said. The Bills won't reveal the rest of their grade scale. But basically, 5.5 to 5.99 is a prospect who has a chance to become a high-quality NFL player. A prospect at 6.0 to 6.49 should become a player with Pro Bowl potential. Those at 6.5 and up have a good chance to be Pro Bowlers or All-Pros.
The visits to college campuses in March are used to do further interviews with players and coaches and to fill in some blanks -- on heights, weights and speeds -- especially on juniors leaving college early to enter the draft.
Bills scouts visit more than 100 schools in March, searching high and low.
The Bills, for instance, made visits last month to tiny schools such as Lambuth College in Jackson, Tenn., and Lenoir-Rhyne College in Hickory, N.C. At Lambuth they saw a receiver named Ron Dixon ("He was No. 3 in the early part of the season but he switched to No. 27 after the equipment bus burned up," Adams said). At Lenoir-Rhyne they saw a defensive end and ex-Marine named John Mylum.
"Some schools, like Penn State, for instance, don't let Blesto (a national scouting service) time their guys in the spring," said Doug Majeski, who scouts the Northeast and Midwest. "So we have no confirmed times on their guys. We go in there and time them in March."
"The advantage of this visit is being able to meet the guy and shake his hand and sit down and visit with him," Adams said. "The coaches are a little more relaxed at this time of year. Their season isn't going on. We'll sit down with them and learn a little more about their guys."
All of that sets the table for the final evaluation meetings, which began last week.
"For every player we evaluate, we go in chronological order of the people who scouted him," Adams said. "The first exposure to the player, let's say, was by Doug Majeski. Doug gives his report. Then the second scout who saw him gives his report. Then the position coach, Charlie Joiner, gives his report. Then the coordinator, Joe Pendry, gives his opinion, then it comes to me."
After everyone gives his opinion, a final grade is put on the player, and it's on to the next prospect.
"If I have a 5.6 grade, and somebody's got a 5.7, and he's got a 5.8, then we've got to find out what that guy is," Adams said. "We're all right there in the same ballpark, but that's got to be worked out."
With all the cross-checking that's done, it's impossible to give credit to one scout or coach for drafting a Thurman Thomas or a Henry Jones, for instance.
"We make jokes about that," Adams said. "We facetiously stand up once in a while and say, 'Well that's my guy. I was responsible for him.' And we get a big laugh out of that. There's nobody in here responsible for 'a' guy.
"It's a collective effort of the scouts and the coaches, and that's why we think it works."