Tag Archives: assessment

CollegeQandA asks: Should I be using MOOCs in conjunction with my typical college education?

Should you be doing MOOCs while you’re doing your normal courses?

If you haven’t heard of MOOCs (massive online open courses) then, briefly, they are, as the acronym says, big courses that you can do online.  There are a number of platforms that provide MOOCs such as Coursera, Udacity, edX, and Khan academy.  Universities sometimes are hosted on these platforms or offer their own MOOCs.  The topics covered in these MOOCs are vast and varied. There is a heavy focus on computer science because of the link between MOOCs and technology, but most fields have some course covering almost all the intro courses and some more advanced courses.  These courses can be offered for free, or if there is some sort of qualification, then you pay for the verified assessment and credential.

Highland CowThe bigger philosophy behind the MOOC is that modern technology allows some of the best teachers to provide instruction at a mass scale beyond the walls of traditional universities and college.  Early research has shown some successes, but other results that even though these courses have massive enrollment, the success rate can be much lower than typical classrooms (in the 5 to 20% range).  Even the successful students are already familiar with the material and self select.

So, are MOOCs going to be useful to you and your education?

It depends.

Let me, first, describe my MOOC experiences.  My first course was a google search course.  The time commitment was around 8 hours and I learned a few ideas to improve my search skills.  Next, I enrolled in one of the earlier Artificial Intelligence courses that I soon dropped out of, because I couldn’t spend the ten plus plus hours a week needed to complete the assignments.  I didn’t do any more online courses for a few years until recently in my research leave where I completed both Learning How to Learn: Powerful mental tools to help you master tough subjects and Mindshift: Break Through Obstacles to Learning and Discover Your Hidden Potential in the last two months (both courses are offered for free on Coursera).  Finally, I enrolled for a course on teaching, but dropped it shortly after enrolling because of the time commitment.

In all of this, my key take away is that time is a huge factor for me, and I need more then a passing interest to commit to a course.  I, personally, am not interested in the online peer community, and my lack of commitment to community (online) is a big loss in this form of education.  Also, the courses I tend to complete have shallow assessments, such as quizes, and I might argue that the skills that I’m learning from these courses are not that complex.  I, however, would highly recommend the two MOOCs listed above to everyone since they provide value regardless of what field you are studying in helping you manage time and learn.

So, should I?

In the bigger sense, I think MOOCs are great opportunities for you in non-semester time (the summer and breaks).  These courses require significant dedication, and unless you have a really light semester of traditional school work, I would stay away from them during your normal times.  The exceptions are:

  • Find a MOOC that parallels one of your traditional courses.  In this case, the MOOC is a potential secondary source of information to supplement your learning.  Personally, I’ve never had a chance to parallel a MOOC with a live course, but I suspect it would be a great opportunity.  If anyone has done this and wants to provide the rest of us with enlightenment, feel free to email me your thoughts and I’ll make a guest  post for the rest of us.
  • Doing a short course in the beginning of the semester when you have a little extra time.
  • Taking a meta-skill course such as Learning How to Learn: Powerful mental tools to help you master or Mindshift: Break Through Obstacles to Learning and Discover Your Hidden Potential since these types of courses will help you develop skills to succeed in your current courses.

Credits: Photo titled: Highland Cow by Mike Davison

Share

CollegeQandA asks: Who assesses the assessors?

Who assesses the assessors?

To rephrase, is there any formal processes that checks if colleges, professors, and higher education are “good”.  Simply, are professors who assess students also assessed?  The simple answer is, yes, and professor assessment happens in a number of ways.

Measuring Tape up close

Promotion, Appointment, and Tenure

As an individual professor the main assessment is one or a combo of promotion, appointment, and/or tenure.

  • Tenure = typically, a decision based on a period of 3 or more years where the institution decides if you will be a member of the institution for the foreseeable future.  The criteria for tenure varies but factors in research productivity, teaching ability, and service to the university.
  • Promotion = this is an institutional decision to promote someone to a higher position.  This is similar to climbing the ladder in large corporations.  Criteria for promotion varies and are similar to tenure, but expectations tend to increase as the ladder is climbed.
  • Appointments = some institutions don’t have tenure and instead use X year appointments.  After X years are complete reappointment is determined based on meeting criteria similar to above.

In each of these individual assessments, a professor or educator is judged on criteria of productivity.  What this means to a student is that a “full professor” or “university professor” means that the person has been promoted to some of the highest levels of the institution, but it might be due to their research productivity more than their teaching ability (and typically this is more often the truth).  Higher ed is not just about learning at the undergraduate level, and a major push is learning at the frontier of human understanding.

Assessment agencies

Beyond the individual, there is also institutional assessments at every level.  For example, engineering departments have the option of being assessed and accredited by ABET.

For these types of accreditation (an award of meeting some standard set by the accreditation agency), programs have to show measurable results on various factors.  For example, what percentage of students are gainfully employed after graduation?  How many students are failing?  These measurements are an attempt to evaluate the institution and the quality of its respective programs.

A world of measuring and setting bars

The reality of assessment – student, professor, institution – is we like to measure things and set bars.  Look at all the school ratings that are generated each year.  With measurements we can compare us to you and you to us.  Is that a good thing?  My feel is that we overly rely on measurement, and the world is too complex to simply measure.

One of our next big philosophical leaps will come from a shift from our love of the simplicity of quantity.

Credits: photo titled: Measuring tape ; by Sean MacEntee

Share

CollegeQandA asks: What will be on the test?

What will be on the test?

This question can also be phrased, “will this be on the test?”

For students, this is an important question.  For teachers, this is a frustrating question to hear more than once.  How do you figure out what’s on the test, but also, turn yourself into a good learner and not just a test gamer?

FRustrated man screaming

Student perspective of tests

Tests are pretty common in courses.  Tests can be one of the main assessments that then are converted into your overall grade in a course.  Students need to get good grades, and therefore, knowing and understanding what is on a test is fundamental to getting those good grades.

Teacher perspective of tests

Learning the material is common in all courses.  Tests can be a good and easier than others assessment that shows if students have learned the course material.  An overall grade shows how well students have learned the material (though I don’t really agree with this).  Testing helps determine a students grade in the course.

Some of education’s bigger perspective

The goal is to learn.  Real understanding can not be easily tested, and the learning process takes a significant amount of time for both teacher and student to achieve and assess.  How can we teach and learn important skills, understanding, and complexities with tests?  We probably can’t, and equally, grades are a poor signal in terms of how each of us will perform in the real world with complexity.

The reality perspective

So, you are going to have tests as a student, and many of us professors will use tests as time-reasonable assessments for what you understand in our courses.  Coming from a perspective of wanting to learn and understand and willingness to work on learning, be curious, and understand the world better will serve you well.

If you need to figure out what is, likely, on the test, then pay attention to your professor in lectures, do the class work (readings, assignments, and projects), and find more senior students to understand past testing trends.  In lecture, emphasis of ideas, points, and skills are strong indicators that the material will appear on a test.  Does the professor underline writing?  Is there a significant number of problems dedicated to a particular idea?  Many professors (myself included) plain and simple say, “and this would be an excellent question on an exam”.  Those signals are strong indicators that the material will be tested in the future.

Also, the past can be a strong predictor of the future.  Students who have previously taken the course will have a feel for what professors will ask on their exams.  Make friends with other students and ask about their experiences.  However, understand that we all have biases and you are the one who does or doesn’t benefit from learning.

Credits: photo titled: frustration; by Rakesh Rocky

Share

CollegeQandA asks: Why are projects so important in College?

Why are projects so important in College?Building Construction

Projects are some of the closest activities you’ll do in College that have some similarity to the working world (that I assume you are trying to join).  A project allows you to demonstrate that you can do something from early conception to close to completion.

There are very few jobs in – taking exams

It’s true.  There just aren’t many jobs out there in taking exams, regurgitating facts, showing the steps on how you solved something, and picking between letters in multiple choice or picking true/false.  The follow up question should be, “Why then do we take all these exams in school?”

Well there are a number of reasons to take exams, but start with this idea that exams are meant to assess how well you have learned a particular set of concepts and ideas.  How should a teacher determine if you understand these ideas and concepts and give you feedback on your learning?  Exams and tests are a way to do this assessment at a reasonable time cost.  For example, with a test that I can create in about 4 hours, have students do in 1 hour, and mark/grade in 8 hours it costs only 17 hours and maybe 52 person hours (for a class of 30 students).  If a machine can grade the test, that time goes down.  So in many cases examinations are the easiest and most efficient (time perspective) method for assessing student knowledge.

For that same class of 30 students, if I spent 20 minutes in an oral examination to evaluate their understanding, it would take 10 hours assuming that I know the right oral questions to ask and probe, there are no delays between students, the scheduling of these oral questions is magically administrated, everyone takes exactly 20 minutes, and so on.  That assessment is harder to implement for the teacher though and the time will always be much greater.

Projects to the rescue

A project in a class is an opportunity to create an activity that can be assessed in reasonable time, but the depth of student work (including creativity, problem solving, communication, doing, and critical thinking) goes far deeper than almost any exam can capture.  Plain and simple, the learners doing the project tend to learn far more since they have to do something and solve all the steps to get it done.

I also like open ended projects since students have the opportunity to create and do something they’re interested in.  For engineers, this might be one of the few opportunities in their lives to work on their creations as opposed to being told what they need to do.

Finally, projects with their depth and unknown challenges they provide mirror more of what people tend to do in the working world.

Is there anything projects can’t do?

The worst thing about class projects is they lack a completion and quality aspect.  For example, a student group starts to do their project, gets close to the end of the semester, and runs out of time.  What do they do?  Submit what they have done and probably get a lower grade for their work.  The same is not true in the real world.  A lower grade for not completing a project or delivering a weak project is getting fired.  So projects lack a more pure assessment in college.

Second, most projects are done in groups again to simulate the “real-world”.  Group work has all sorts of complexities that make it hard to achieve something.  It seems like someone always free loads.  People are hard to work and get along with.

Projects have many other limitations, and are just another activity/assessment that is part of your learning.

Projects are good

Still, each project you have is an opportunity to establish your portfolio.  Projects are the rare opportunities in school to allow you to take some autonomy (see the book Drive by Pinker).  Projects can be lots of fun.  And projects are about the closest to real you can get in college.

Credits: photo titled: Construction; by Stephen Rush

Share

CollegeQandA asks: How important are my grades?

How important are my grades?

Package with eggsI’m from Canada and I like to say, “we grade eggs, but we assess people to give them feedback on their learning”.

I, personally, think public grades as a measurement device of people is a bad idea.  Of course, I’m one of those weird people that is focused on this idea of learning.  From my perfect world perspective, I think that grades can be a summative indicator of your performance on an assessment(s).  As a private feedback measurement a grade is useful in helping the learner have a summary indicator that shows their performance on learning.  The summary grade with detailed feedback on what other aspects of the task that need to be worked on is a useful way for the learner to try and move forward.

Forget your utopia, how important are grades

In a credentialist (not sure that’s a word) based society, grades are important.  The reality is that grades are used by institutions to quickly evaluate people.  For example, a company with an entry level position might receive 300 applications for this job.  One simple way to filter these 300 candidates is to make a grade point average (GPA) mark and cut everyone who hits below that line.  This is needed since it is difficult and time consuming to evaluate all 300 people for a single position efficiently.

In many ways, we use grades as a quick indicator of how a person is performing in topic areas.  From a student perspective, grades are important since they impact what possibilities are available once completing a degree in particular for that first job.

Still important?

The odd thing with your GPA is it becomes almost irrelevant once you get your first job.  This is because your next job or promotion will be based on what experience you have at getting things done.  Or as I like to say, “Can you do things?”

The grade is a very poor measurement of your doing stuff ability.  However, if your job is taking tests, then it is a great indicator.

Going forward

So, yes, grades are important, but learning and doing is much more important for the long game.  The grade is a single measurement signal of how you have performed.  Going forward though, the portfolio is becoming  a more appropriate signal of your ability (and not just for artists).  In college, every major project you do, create, and build is a better signal that demonstrates what you can do.  This includes your activities outside traditional class and you should be curating a web presence to host this portfolio.

If your GPA is not as good as you need it to be, then not all is lost.  If you have a portfolio of what you can do, you can work your way into entry jobs at smaller and lesser (in the eyes of the public) institutions.  People who can learn and do things are valuable.  So, it’s not all about grades, but good grades tend to correlate to people who learn, work hard, and do things.  And that’s the chicken back to the graded egg.

Credits: photo titled: Eggs; by bunnicula

Share

CollegeQandA asks: What is the best way to study?

Let’s start with the assumptions and definitions

Definitions:

  • Test – an assessment technique to evaluate student learning and provide feedback on what has been learned – yeah I know, optimistic
  • Test dump – short term studying period, taking the test, and then forgetting everything afterwards
  • Studying – time spent towards learning ideas, concepts, and knowledge
  • Problem – a question, challenge, or need that may or may not have a solution

For the sake of this discussion I will make some assumptions that if you do not hold them, then the advice is unnecessary.

Assumptions:

  • The goal of studying is to learn
  • The goal of assessment is to help the student understand what they have and have not learned
  • Problems encountered in undergraduate courses at the first and second year are mostly solvable

How to study

studyAs stated above, studying is a method of spending time to learn about an area.  From what we know about learning, the actor (us) needs to activate pathways within our brain to create and refine the networks associated with that area of knowledge/skill (The Art of Changing the Brain: Enriching the Practice of Teaching by Exploring the Biology of Learning).  Also, it would be great if things learned for one thing are transferable to other areas.

This all implies that you, the actor, need to study by doing, reflecting with feedback, and repeat.  So, depending on the course this can be done in a number of ways.   For example, to write better, you need to write, edit, rewrite, edit, … maybe with a person to help you refine, rewrite …

Problem based courses – Math, Physics, etc.

For problem based courses, which are courses people tend to find most challenging, the doing is repetitive practice of problems spread out throughout the course.   This comes from a meta-study by Dunlosky et. al. (2013) in which they looked at the best ways to learn material in college as a summary of many other studies.  Throw out your highlighters.  Stop rereading passages.  Focus on practicing problems that are assigned and will be similar to those on the assessments.

Just like learning a new sport skill, learning an academic topic means practicing.  For courses where the major assessment is tests and exams, the act of practicing problems (both the mechanics and the understanding of material) is the best doing to study successfully and succeed in these types of courses.  Secondly, make sure you are doing this throughout the semester, and repeating problems that you already think you know how to solve.

Credits:  creative common photo from The Master Shake Signal; titled: Clean Study Bag

Share