that has been established, but not firmly enough to warrant us in recommending that they be used exclusively for admission. We should not recommend that.

We have made another attempt in starting to develop a test specially adapted for engineering students. This examination that I am distributing now is the first attempt, and you will no doubt notice that it is a crude one. I think that a further attempt of that sort should be made by pooling the very best judgment, such as can be obtained from this Association. And before we carry out an extensive program of that sort it would be well if the committee or committees in charge of this work could get together and devise these tests or at least go over the material before it is printed for next year's program. I should like to extend an invitation to those of you who want to use our material, to make use of the tests that we are devising for the September issue. We should be glad to have you coöperate with us in devising tests, especially tests for engineering students, and then we might give these tests on some day in September after the students have been admitted and started their college work and later in the spring of the following year collect our data for the test scores and the academic standing of the students to determine the diagnostic value of the tests.

At the Carnegie Institute our plan is to give all of the entering students a complete series of intelligence examinations as their first academic exercise. Admission will not be determined by that test. In other words, we want to test the tests before we test the students by means of them.

We try to get the right attitude of the student by giving the student the examination on his very first day in college. He is then in a better attitude to take the test and he will go after it and do his best; whereas if you give the test later on, he may not do his best because he realizes that nothing depends on it.

That is not as serious a difficulty as it might seem. I find it is easy enough to get the good will of the student by which they will do their very best.

not like to exclude or retain a test because somebody finds that this or that particular student failed in the test or made good. We want the records of all the students to count in determining the value of the test, so we plot what we call a scatterdiagram, showing these variables. One variable is the test score objectively determined. We try to arrange the tests so they can be scored by stencils and by clerks who may not even understand the content of the test. That is the ideal form of test, in which the test score would be the same no matter who scores it.

And then we plot the scholarship standing of the student. We have two different forms of criteria for determining the academic standing of the student. We have the scholarship pooled, the average scholarship in all the subjects, and also the scholarship in the individual subjects. Besides that we ask the instructors to rate each student on a scale of from one to ten, and we use these instructor's estimates as criteria for checking up our tests.

We have then these instructor's estimates, and also the scholarship grades. What we get is a diagram with one point for each student. (Diagram drawn on board.) Here we have the score in the test. Here we have the scholarship standing of the student or the average instructor's estimates. If the test is a good one, then the points will be arranged diagonally in that chart-that is, the higher the score in the test, the higher will be the rating by the instructor.

Now, there are exceptions. There are always exceptions. If the test is a poor one, then those points scatter all over the diagram; and in order to get a definite statement of the diagnostic value of the test we calculate the correlation coefficient -the coefficient used by actuarial statisticians. We use this regression line in studying our diagrams so as to avoid personal opinion.

Then, we also study individual cases, those cases which occur in this quadrant and in that quadrant, and we interview those students, and make further inquiries of the instructors

gram on board.) Do you not find there is something of that kind?

Professor Thurstone: This will be scholarship and this will be grade?

Professor Magruder: And rank.

Professor Thurstone: In the test?

Professor Magruder: Have you compared them with your results?

Professor Thurstone: We have in a preliminary way. We have made a comparison between the engineering test and the instructor's estimates based on ten weeks' acquaintance. They give about the same soefficient that we would get from the intelligence tests after ten weeks' acquaintance. The coefficients are not quite as high as those we obtained after one year's acquaintance with the students. Both of the lines, based on ten week's acquaintance, are practically the same. But they may shift when we come to compare the grades in the tests with the scholarship after one year. That I can not answer yet.

Dean Butler: Do you make any difference in giving these psychological tests between the man who answers all his questions as far as he goes and does not finish up and the man who goes clear through and does not answer so well?

Professor Thurstone: Yes, we do that in this way: We score the tests in several ways. We will score a test by counting only the number of right responses in a given time. We give a time limit say of thirty minutes for the examination, and then we score the papers by counting only the number of correct responses, and then by counting. the number of correct responses minus the number of errors made, and the number of correct responses minus half the number of errors made. As a matter of fact we can determine by the method of partial correlation how much to count off each error. It varies from one test to another. We do that with other tests. The engineering test which you now have is one of the new tests, and I cannot make this evaluation until I get the schol

arship records at the end of this year. However, I have made that evaluation on the first two pamphlets that I distributed, and there the test scoring method is to count simply the number of correct responses in the thirty minutes allowed. Practically nothing is gained by counting the number of errors.

C. F. Allen: Mr. President, I would like to inquire in what respect this test of engineering aptitude differs from a set of questions given by an engineering professor who thoroughly understands his business?

Professor Thurstone: That is a perfectly, legitimate question, and it is very easily answered. The fundamental difference between a mental test for engineering students and a test which is given at the end of his course would be this, that the test given at the end of his course depends largely on the memory of the detailed content of that course. It also depends of course on the memory of the procedure and the use of empirical formulæ and so on. In an examination for engineering aptitude the bright and talented student who has been away from high school three or four years and who only retains the fundamental concepts of physics, and perhaps the very rudiments of his algebra, would make a high score on an aptitude test because it should not require him to use material which is simply dependent on the memory of detailed facts. He is only called upon to use his good common sense in solving problems which are related to engineering. He has to use only the fundamental concepts of his high-school work. There is a very fundamental difference between those two; and that is exactly what we are driving at. If we are to use the tests for freshmen, you cannot very well test them on engineering knowledge that they have not yet acquired.

Professor Allen: Mr. President, the question may still arise as to whether the engineer, who understands his business, is going to ask questions which largely depend on knowledge. I would like to ask a question which may be irrelevant, whether this test for intelligence has ever been given to members of the faculty.

Professor Thurstone: It has, but not officially!

AN EXPERIMENT IN VOCATIONAL INTELLIGENCE TESTING OF FOURTH YEAR MECHANICAL ENGINEERING

STUDENTS.

BY WM. T. MAGRUDER,

Professor of Mechanical Engineering The Ohio State University.

During the recent 1919 inspection trip for electrical and mechanical engineering students of the Ohio State University, the remark was made by an engineering executive and employer that his company had been trying-out some of the college men that had been sent to them with the result that they found that, in a number of cases, those students who were most highly recommended by their professors as being among the very best men in the class were found to be correspondingly deficient when it came to work on the testing floor, and that they seemed to lack that quality which he termed "mechanical sense" and which used to be known by the name of "gumption" or "intuition," but which is now dignified by the titles of "vocational intelligence" and "engineering aptitude.

The reading public has heard so much of "mental tests,' "psychological tests, "intelligence tests" and the like, since the psychologists showed the army how best and most easily and systematically to select men for the specific job for which men were wanted, that the idea of applying some such test to those students who were about to be graduated and recommended to the employers was not as novel as it would have been five years ago. The novelty of the so-called "army tests" lay not so much in the tests themselves as in the systematic and wholesale way in which they were administered and the satisfactory results which were obtained; for, it must be remembered that many people have been applying