I'm leaving my current job to go to another company so I'm helping my current boss interview candidates to replace me. I drafted a 40 question test from scratch covering Active Directory, Exchange Server, Terminal Server, SQL Server & general networking questions that I thought we could use to gauge candidate's skill levels. My problem is that the first two candidates completely bombed my test (25% or less correct), although they had MCSE, CCNA, Bachelor's Degrees in Information Technology or similar credentials + 4 or more years experience

39 of my 40 questions I made up in my head, without any reference material, so I figured it was a safe bet that experienced administrators would be able to answer a lot of my questions w/o too much difficulty. Anyone willing to look over my test to tell me if I'm being to difficult, or if the first two candidates just don't know their stuff

Here is the list of qualifications we asked for

Must have thorough knowledge of Microsoft Server Software and Services (i.e. Exchange, SQL Server, Terminal Server and Active Directory), Windows NT based Client Operating Systems, Network Security, and end user software like Microsoft Office, including Access and other database software. Must have experience performing hardware installation, configuration and repair, i.e. hard drive, RAM or CPU replacement, network cabling, router, switch, firewall, workstation and Server (with RAID Storage Systems) configuration

Test questions here
http://workthin.com/DOC/NetworkAdmin...houtAnswers.do

Thanks to anyone willing to help me out. You may email me your response via my website (support request). I'd also be willing to furnish the answers to anyone who wants them

Patrick Rous
Microsoft MVP - Terminal Serve
http://www.workthin.co

"Patrick Rouse [MVP]" <(E-Mail Removed)> wrote in message
news:1519C216-263C-476F-8E13-
> 39 of my 40 questions I made up in my head, without any reference
material, so I figured it was a safe bet that experienced administrators
would be able to answer a lot of my questions w/o too much difficulty.
Anyone willing to look over my test to tell me if I'm being to difficult, or
if the first two candidates just don't know their stuff?

I didn't look through the entire document but I would say that your
questions are way too specific. Very competant admins might not know the
answer to your questions. I think it would possible for most admins to put
together a similar doc that you would only get 25% on simply because you
have not run into those issues in you line of work. Just say there are
100,000 different facts that are along the lines of your questions and any
network admin would know the answer to 5,000 of them off the top of his
head, but everyone knows a different 5,000 Q&As, that means anyone could
write up a series of questions that others would have trouble answering.

BTW, you got question 13 wrong. :-)

Michael Culley

"Patrick Rouse [MVP]" <(E-Mail Removed)> wrote in message
news:1519C216-263C-476F-8E13-(E-Mail Removed)...
> I'm leaving my current job to go to another company so I'm helping my
current boss interview candidates to replace me. I drafted a 40 question
test from scratch covering Active Directory, Exchange Server, Terminal
Server, SQL Server & general networking questions that I thought we could
use to gauge candidate's skill levels. My problem is that the first two
candidates completely bombed my test (25% or less correct), although they
had MCSE, CCNA, Bachelor's Degrees in Information Technology or similar
credentials + 4 or more years experience.
>
> 39 of my 40 questions I made up in my head, without any reference
material, so I figured it was a safe bet that experienced administrators
would be able to answer a lot of my questions w/o too much difficulty.
Anyone willing to look over my test to tell me if I'm being to difficult, or
if the first two candidates just don't know their stuff?
>
> Here is the list of qualifications we asked for:
>
> Must have thorough knowledge of Microsoft Server Software and Services
(i.e. Exchange, SQL Server, Terminal Server and Active Directory), Windows
NT based Client Operating Systems, Network Security, and end user software
like Microsoft Office, including Access and other database software. Must
have experience performing hardware installation, configuration and repair,
i.e. hard drive, RAM or CPU replacement, network cabling, router, switch,
firewall, workstation and Server (with RAID Storage Systems) configuration.
>
> Test questions here:
> http://workthin.com/DOC/NetworkAdmin...outAnswers.doc
>
> Thanks to anyone willing to help me out. You may email me your response
via my website (support request). I'd also be willing to furnish the
answers to anyone who wants them.
>
> Patrick Rouse
> Microsoft MVP - Terminal Server
> http://www.workthin.com

While managing a team of a dozen professionals, I found myself much in the
same situation as you now do. On two occasions I hired the wrong person:
Good credentials, lots of relevant experience, said all the right words
during the interview but turned out to be a liability rather than an asset.
I subsequently introduced a real-life test, and was able to attract a number
of excellent people.

While your set of questions is highly relevant to the qualifications you
specify, I think that the answers you get reveal far more about the
***current*** knowledge of the applicant than about his/her potential. In
other words, someone getting 40 out of 40 correct would be your twin
brother: He knows the job inside out. Is this really what you want? Every
new job should be a challenge, an opportunity to grow, to acquire new
skills, to pick up new technologies. Every job is scary at first, because of
one's level of ignorance, but if the ingredients are right then success will
follow. A person who answers every question correctly would either get bored
with the job within two months, or he has prepared himself with the link you
quote (which shows the name & location of your organisation).

If this was my show then I would probably split the questions into three
parts:

Part 1: Knowledge test. Pick a dozen of your most essential questions, i.e.
questions that are fundamental to the topic you're exploring.

Part 2: Test the candidate's potential. Compose a number of questions that
explore how the applicant would deal with the introduction of a new
technology. What steps would he take, what resources would he apply, has he
ever been in this situation before, what did he do?

Part 3: Test the candidate's resourcefulnes. Rather than expecting him to
know the various port numbers for specific services by heart, see if he can
find what port is used by Remote Desktop, while seated at a PC with an
Internet connection. Let him check Google for the maximum reliable run of a
CAT5 cable.

Splitting your questions like this will allow you to gauge each candidate
not only by his current detail knowledge but also by his ability to grow and
by his resourcefulnes.

I would also agree that questions are too specific.
First about my apeciality - I am mainly windows application and web
developer with much experience in network administration, security and
hardware.
Here is one article that I like much on this theme:
http://joelonsoftware.com/articles/fog0000000073.html
On what operation systems can be installed whish version of Exchange...
it is easy to just look at the specification. Why not ask such specific
question, give the people access to Internet and see how fast and acurate
they can find answers and where they will look at - this is how they will
work anyway.
I think that questions from 5 - 10 are importand ones. About Q10 - they
are 5 not 6 disks.
Q11 is ambiquous - both c and d...
Q12 is dumb one. 13 is good one but port 20 is little bit confusing for
FTP - better ask for 21.
Q14,15 are confusing. 16 is strange one - probably you ask for a) but is
strange anyway.
Next ones I have just look at and seem very very specific. It is
possible that they are really needed to be known in order to sucessfully
accomplish these tasks but the fact that you remember specific cases that
probably are happened to you talks that you had hard time with them )

Bojidar Alexandrov

Patrick,

Congratulations on your new job. I hope that everything will be super duper
in your new challenge ( yes, I really did write 'super duper'! ).

I looked at the document and went through it fairly quickly. I somewhat
agree with the other posters. Probably 17 to 20 of the questions should be
common knowledge for someone who has done this job for a couple of years.
So, I would anticipate that someone should - at worst - be getting in the
50% range.

I really agree with Pegasus. I like the idea of letting people find the
answer ( via Google, the appropriate Knowledge Base or News Group,
whatever ) as we all know very well that you can not know everything that
there is to know in this field. As long as someone has a core competency
( aka a very solid foundation ) then there should not be any problem filling
in the blanks. On top of that I firmly believe that everyone has his / her
own way of doing things, that everyone has his / her own style. There will
never be another Patrick where you currently work. If it takes someone
three hours to find the maximum reliable length that you can use for Cat 5
then maybe this person is not the right person!

I would have trouble with the SQL / Access questions that were on your test
but not with much else ( probably because I have never touched SQL ). I do
not know if I agree with the 'way to specific' comments. I think that the
vast majority of your questions are indeed real life situations. I think
that the important thing would be not to simply see if they can add two and
two and come up with four but to see how they came to four ( or whatever
their answer might be ). I remember back in high school Calculus that our
teacher did not really care about the answer ( well, Dr. Woodsmall did care
about that! ). He was much more interested in how you arrived at the
answer. If you did a simple 'addition' error incorrectly somewhere along
the line and that threw off everything from that point forward you did not
get the answer marked wrong, you simply lost a couple of points for 'dumb
addition'. Try to see where their 'dumb addition' errors and then determine
if those errors can be corrected or if they just can't add!

Anyway, good luck to you in your future endeavors.

Cary

PS - It is A L W A Y S those dang printer drivers ;-)

"Patrick Rouse [MVP]" <(E-Mail Removed)> wrote in message
news:1519C216-263C-476F-8E13-(E-Mail Removed)...
> I'm leaving my current job to go to another company so I'm helping my
current boss interview candidates to replace me. I drafted a 40 question
test from scratch covering Active Directory, Exchange Server, Terminal
Server, SQL Server & general networking questions that I thought we could
use to gauge candidate's skill levels. My problem is that the first two
candidates completely bombed my test (25% or less correct), although they
had MCSE, CCNA, Bachelor's Degrees in Information Technology or similar
credentials + 4 or more years experience.
>
> 39 of my 40 questions I made up in my head, without any reference
material, so I figured it was a safe bet that experienced administrators
would be able to answer a lot of my questions w/o too much difficulty.
Anyone willing to look over my test to tell me if I'm being to difficult, or
if the first two candidates just don't know their stuff?
>
> Here is the list of qualifications we asked for:
>
> Must have thorough knowledge of Microsoft Server Software and Services
(i.e. Exchange, SQL Server, Terminal Server and Active Directory), Windows
NT based Client Operating Systems, Network Security, and end user software
like Microsoft Office, including Access and other database software. Must
have experience performing hardware installation, configuration and repair,
i.e. hard drive, RAM or CPU replacement, network cabling, router, switch,
firewall, workstation and Server (with RAID Storage Systems) configuration.
>
> Test questions here:
> http://workthin.com/DOC/NetworkAdmin...outAnswers.doc
>
> Thanks to anyone willing to help me out. You may email me your response
via my website (support request). I'd also be willing to furnish the
answers to anyone who wants them.
>
> Patrick Rouse
> Microsoft MVP - Terminal Server
> http://www.workthin.com
>

Honestly if someone asked me questions like this I would probably walk out of
the interview if I didn't feel like shredding the person asking the questions
instead. Most of this stuff is stuff you look up when you need it or is just
plain silly to ask. It would be a sign to me of someone who I am not sure I
would want to work for/with as the emphasis is on the wrong thing.

You should not focus on what someone knows, but what they are capable of
figuring out or doing. The answers to your questions can be memorized and
someone could know them right there which means they might be right for the job
right now, what about a year from now when something new and different hits. On
the flip side, I don't want someone working for/with me that has simply proven
they can memorize stuff, I want them to be able to figure things out or know how
to find answers.

Questions along that line are much harder to come up with and for a lot of them
there are no single right answers. I would ask things along the line of "How
would you tackle a problem like this or that?". Where would you go look for
answers. Do you use network sniffer software? If so, what software? What are
some issues you figured out? What annoys you about exchange? What annoys you
about AD? How do you feel about patch management? Etc. The responses should
give you a feel of the actual knowledge and capability versus what is memorized
from a book or newsgroups. Basically communicate with the people and get a feel
for them. A knowledgeable admin will pick out other knowledgable admins.

I agree that a lot (most) of these questions are too specific. Many admins I
know that have been forced into a position of interviewing people get into a
****ing battle with the applicant to try and show who knows more. That shouldn't
be the goal however if one of my junior admins brought me this list of questions
that is what I would determine it to be. You should be looking for someone with
the capability to handle the position that exists now and what unknown thing it
can evolve into.

On top of that, you would probably get some arguments over what you think the
answers are to your questions though I am sure you feel your answers are all
correct (and could be in your environment) so any argument would be considered
as wrong in your scoring. There were several questions in there that I am sure
our answers wouldn't align. The situations being described should have been
prevented in the first place (IMO) or they are only applicable in small
environments where my experience is with Global Enterprise Systems comprising
tens of thousands of users or hundreds of thousands of users.

If I actually sat through that interview and you let me finish, my questions
back to you would have been about security because from the direction of the
questions I would be very leary of the environment and how secure it was.

Giving specific questions with specific answers like that doesn't work in only
letting good people become MCSE's, why would you think you can come up with
specific questions with specific answers to do so?

The last few people I have worked with would never have gotten the jobs they got
with me had they had to answer a question list I made up like this concerning
our environment. I doubt there is anyone that could have gotten the job, maybe
even me. However the guys in those slots are really good and I am glad they are
there.


joe


--
Joe Richards Microsoft MVP Windows Server Directory Services
www.joeware.net



Patrick Rouse [MVP] wrote:
> I'm leaving my current job to go to another company so I'm helping my current boss interview candidates to replace me. I drafted a 40 question test from scratch covering Active Directory, Exchange Server, Terminal Server, SQL Server & general networking questions that I thought we could use to gauge candidate's skill levels. My problem is that the first two candidates completely bombed my test (25% or less correct), although they had MCSE, CCNA, Bachelor's Degrees in Information Technology or similar credentials + 4 or more years experience.
>
> 39 of my 40 questions I made up in my head, without any reference material, so I figured it was a safe bet that experienced administrators would be able to answer a lot of my questions w/o too much difficulty. Anyone willing to look over my test to tell me if I'm being to difficult, or if the first two candidates just don't know their stuff?
>
> Here is the list of qualifications we asked for:
>
> Must have thorough knowledge of Microsoft Server Software and Services (i.e. Exchange, SQL Server, Terminal Server and Active Directory), Windows NT based Client Operating Systems, Network Security, and end user software like Microsoft Office, including Access and other database software. Must have experience performing hardware installation, configuration and repair, i.e. hard drive, RAM or CPU replacement, network cabling, router, switch, firewall, workstation and Server (with RAID Storage Systems) configuration.
>
> Test questions here:
> http://workthin.com/DOC/NetworkAdmin...outAnswers.doc
>
> Thanks to anyone willing to help me out. You may email me your response via my website (support request). I'd also be willing to furnish the answers to anyone who wants them.
>
> Patrick Rouse
> Microsoft MVP - Terminal Server
> http://www.workthin.com
>

>Must have thorough knowledge of Microsoft Server Software and Services (i.e. Exchange, SQL Server, Terminal Server and Active Directory), Windows NT based Client Operating Systems, Network Security, and end user software like Microsoft Office, including Access and other database software. Must have experience performing hardware installation, configuration and repair, i.e. hard drive, RAM or CPU replacement, network cabling, router, switch, firewall, workstation and Server (with RAID Storage Systems) configuration.
>
>Test questions here:
>http://workthin.com/DOC/NetworkAdmin...outAnswers.doc
>
>Thanks to anyone willing to help me out. You may email me your response via my website (support request). I'd also be willing to furnish the answers to anyone who wants them.

To start, many of the questions are reciting facts you could always
look up if you ever needed them. I hate those kinds of employment
tests, because it indicates the employer really doesn't know what they
need done. Does your job *really* require knowing how many channels
are in T1? Or the maximum length of an ethernet run?

Beyond that, some of the questions actually can't be easily answered
with your answers. They depend on the environment as much as the
straight Microsoft book-learning issues. Such as your question 25:

"Users can not print to any network printer. The network printers are
setup as TCPIP printers on a Windows Server. Other network functions
on the server are working normally. What is the first thing you
should do to resolve the situation:"

The answer may very well be "Plug the switch all the network printers
are attached to back in and unplug the coffepot like you meant to."

Your SQL questions cover administration and programming, which may be
separate functions in a user's past shop. Your Exchange questions
cover installation parameters (What OS will xxx run on, what needs to
be installed for it to function, etc.) Unless you expect the new guy
to install Exchange fifteen minutes after he starts work without
access to the manuals or an internet connection, these are questions
nobody *needs* to know the answers to. You ask a licensing question
about Microsoft Office and it can't really be correctly answered,
because it depends on the licensing program you're running under as
well as policies of the organization beyond licensing. For example,
we over-license several products, including Office, since it's
adjusted annually and our work environment may change daily.

And the killer that would make me walk is any OSI model question.
When I see questions about the OSI model, I know the employer has no
clue about the job they're hiring for. Except for network analysts
and engineers I have yet to find anyone who uses the OSI model in
real-world scenarios.

You're looking for someone with a lot of depth across a broad array of
technologies, and it's going to be *very* hard to find someone if a
test like this is used as an exclusion. But, there's really only one
type of applicant you want to hire after giving this test anyway.
That's the one who comes back and says "I've written answers for most
of these, but let's discuss what you'd do and why you do it that way
so I can get a feel for what you're looking for in me performing your
job functions."

Jeff

That would be my first reaction, also. My skills are not in memorizing
arcane facts and taking tests. They are in analyzing and resolving problems
and learning new skills. When I started 2 1/2 years ago (after 28 years at
IBM), I knew nothing about AD, Exchange, VoIP, or Citrix MetaFrame. Now,
people actually think I'm an "expert" (hey who am I to tell them
otherwise??). If I need facts, I can look them up. Knowing what OS can
support Exchange 2003 or how many channels are in a T1 line are worthless to
a customer who can't print from Outlook to a Citrix connected printer.

Frankly, I'd be real concerned about hiring anyone with all the requisite
skills & experience that those job requirements and test indicate they need.
It's going to be hard to retain someone who can walk out the door and
increase/double their salary without missing a paycheck.

I support a Hospice in upstate New York and we've hired two people over the
past 18 months. In both cases, it was more important that the person be
someone who had a good background & experience, but just as important was
whether they were a team player, a person who could learn and someone who
was willing to listen. We ran into at least one candidate in our last round
of interviews who seemed to have the level of experience that the original
poster required. Neither one of us put him on the short list. Qualified? You
bet your ass. Team player? not a chance. We knew he was history as soon as
the job market improved.

--
Regards,
Hank Arnold

"Joe Richards [MVP]" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> Honestly if someone asked me questions like this I would probably walk out
of
> the interview if I didn't feel like shredding the person asking the
questions
> instead. Most of this stuff is stuff you look up when you need it or is
just
> plain silly to ask. It would be a sign to me of someone who I am not sure
I
> would want to work for/with as the emphasis is on the wrong thing.

"Patrick Rouse [MVP]" <(E-Mail Removed)> wrote in message
news:1519C216-263C-476F-8E13-(E-Mail Removed)...
> Must have thorough knowledge of Microsoft Server Software and Services
> (i.e. Exchange, SQL Server, Terminal Server and Active Directory), Windows
> NT based Client Operating Systems, Network Security, and end user software
> like Microsoft Office, including Access and other database software. Must
> have experience performing hardware installation, configuration and
> repair, i.e. hard drive, RAM or CPU replacement, network cabling, router,
> switch, firewall, workstation and Server (with RAID Storage Systems)
> configuration.

I suspect you are simply expecting too much. When you started your present
job, what do you think your rating would have been on the test? Chances are,
you have learned quite a lot of these things on the present job through
trial, error, and looking things up on Google.

My suggestion would be to dump all but the 2nd last page of the test. Let
people tell you what they're
good at, and then ask questions tailored to what they say they are good at.
Face to face, so you can judge how they think about it - because afterall,
we use Google/whatever most of the time.

My suspicion is that what your company needs is someone who is capable of
learning more than they need someone who can memorise facts. Find someone
who has good experience in the areas *most* important to your company and
who can also learn the rest as and when it's required.

Regards,

John

It seems the general consensus here is that the test is far too specific and
too long. I agree that the question set is pretty specific in places but
some of the questions are need-to-know stuff. I see about half the questions
as either who cares or who cares, I can look it up in 2 min or less and have
a full KB article describing the steps to completion. That said here's a
breakdown of the questions as I see them [if you care, which I guess you
should since you asked :-)

1. Should know this
2. Look it up
3. See 2
4. Who cares, look it up when necessary
5 & 6. See 4.
7. Fair, but should be referenced before install as part of planning
8. Who cares
9. See 8.
10, 11 & 12. Yank these.
13. Fair question and one a net admin should know
14 & 15. See 4.
16. Answer = Windows.
17 Fair question
18. That's the Net Manager's job to know that
19. Fair permissions question. A candidate should get this one.
20 & 21. Yank these
22. Fair enough
23 & 24. Yank
25. Fair question
26. See 25.
27. Pleeease
28 & 29. Yank
30. Yank
31. Good question, too specific
32, 33, 34, 35 & 36. Keep 33 yank the rest
37 & 38. Who cares
39. Look it up
40. Answer = Fire him for being an 1d10t

You've got a right to test a candidate's knowledge but reading through this
list gave me flashbacks to writing all those annoying certification exams.
More emphasis on team, research and think-on-your-feet-creative skills and
less on the heavy tech.

HTH
Reg
"Patrick Rouse [MVP]" <(E-Mail Removed)> wrote in message
news:1519C216-263C-476F-8E13-(E-Mail Removed)...
> I'm leaving my current job to go to another company so I'm helping my
current boss interview candidates to replace me. I drafted a 40 question
test from scratch covering Active Directory, Exchange Server, Terminal
Server, SQL Server & general networking questions that I thought we could
use to gauge candidate's skill levels. My problem is that the first two
candidates completely bombed my test (25% or less correct), although they
had MCSE, CCNA, Bachelor's Degrees in Information Technology or similar
credentials + 4 or more years experience.
>
> 39 of my 40 questions I made up in my head, without any reference
material, so I figured it was a safe bet that experienced administrators
would be able to answer a lot of my questions w/o too much difficulty.
Anyone willing to look over my test to tell me if I'm being to difficult, or
if the first two candidates just don't know their stuff?
>
> Here is the list of qualifications we asked for:
>
> Must have thorough knowledge of Microsoft Server Software and Services
(i.e. Exchange, SQL Server, Terminal Server and Active Directory), Windows
NT based Client Operating Systems, Network Security, and end user software
like Microsoft Office, including Access and other database software. Must
have experience performing hardware installation, configuration and repair,
i.e. hard drive, RAM or CPU replacement, network cabling, router, switch,
firewall, workstation and Server (with RAID Storage Systems) configuration.
>
> Test questions here:
> http://workthin.com/DOC/NetworkAdmin...outAnswers.doc
>
> Thanks to anyone willing to help me out. You may email me your response
via my website (support request). I'd also be willing to furnish the
answers to anyone who wants them.
>
> Patrick Rouse
> Microsoft MVP - Terminal Server
> http://www.workthin.com
>