T&P Reviewer Feedback 2017 Pilot

Kauai CC

Eligibility

• 2 morning reviewers generally glance at this section and they leave it to the chancellor/coordinator to verify that they're eligible
• straight forward
• reviewer looks closely at eligibility to make sure that they're eligible to apply (applicant didn't meet the MQs)

Dossier

• found it easier to navigate than interfolio
• applicant added documents unnecessarily
  • throwing everything in to supporting documents to cover all bases and that not what reviewers want to see
• one wanted the supporting docs or links side by side with narrative
  • training point: links open in a new tab so show them how to view them side by side
• much faster to review
  • used PCs
  • it was easy to find things and references to appendix
• AR (afternoon reviewer) liked the categories because when he went for tenure he had no clue what to do
  • easier to find information
  • pleasantly surprised that it was a good experience because he's anti tech
  • applicant organized her narrative in a clear and concise way so he's recommend using that as a template
  • opening tabs worked ok; he went back to main T&P site
  • did it on a laptop
  • best thing was being able to do it at home
  • use of the hyperlinks made it really easy plus it was well written
  • he thinks pagination in general is a good idea
  • uploading report (he was the chair) was piece of cake
    • he understood Defer for DPC
    • person was going from rank 4 to 5 and they were 3s so they just did strengths and weaknesses and then pass it to the next level
  • process is clean since this is the first pass at it
  • no frustrations; had more frustrations with paper documents and trying to find supporting documents
  • additional information for committees
    • email sent from chair to chancellors office
  • jotted down notes, when they met, they went through the document and discussed
    • other committee members did the same and it worked fine for them
    • prefers his own notes vs having it online
  • allow applicants to view exemplary dossiers
    • how to do it efficiently
    • Erin forgot to provide student outcomes so they requested that information but it wasn't a glaring mistake
      • it was a risk for Erin and took guts because she was going for tenure, not promotion or contract renewal
  • would he do it online or paper when he's up for promotion

Experience all good; no frustrations

• features they liked
  • easier to access 
  • didn't have to deal with the binders
  • narratives were more concise than the paper dossiers

secret ballot for TPRC

• convener used survey monkey

feedback mechanism or popup survey when they're done reviewing

• reviewers couldn't remember because it was so long ago that they reviewed; providing feedback right after their experience would have been great

agree that applicants should be trained more than reviewers

reviewer training

• how to navigate dossier
• review narrative and supporting document side by side
• how to upload recommendation report

Kapiolani CC

Morning session

Eligibility 

• what was missing?
  • list of courses taught, list of assigned times
    • it's in the dossier but not required since those sections can be deleted

Dossier

• one guy couldn't copy and paste when taking notes to quote from their dossier
• asian guy likes this format better than paper
  • becasue he was TPRC chair he uploaded it early and then wanted to make a change but couldn't because he pushed a button
    • add a warning that once you submit, you won't be able to access this screen
• links were easier to follow than interfolio
• dossiers were highly organized
• choice of software was not an issue
• drive home the point of making it easy for the reviewers
• roman numerals or numbers for each section
• mostly laptop and desktop users

one was surprised that the recommendations were downloadable but they would need to be stored in google to be able to do that

• can we prevent people from downloading?

Louise wasn't sure what she needed to do at the chancellor level but she figured it out eventually

• little confusing; make clearer and more intuitive
• she had to upload chancellor report; should it be required for chancellor?

Collette is with a D&D committee at Windward

• mariko did a great job on her dossier

Frustrations

• none
• have the coordinator be able to push the document back a level

Really liked

• easy than flipping through binder
• linking worked well compared to static interfolio website

Parallel processing

• needs to be union negotiated
• Louise got busted for reviewing a dossier digitally sent by the applicant and the union said that's not allowed because it's not in the contract

Commenting on google docs

• is it possible to gather all comments at the end (yes by hitting the comments icon)
• does it track the name of the commenter (yes)

one liked meeting new poeple on the TPRC so it was a social observation

• this will morph into people not meeting but such is the new world of video conferencing

Julie Rosado - I thought the on-line application went smoothly.  It was user friendly for the assessor and I have no suggestions for improvement. 

Lexer Chou - I think it worked well…it wasn’t a problem for me and it was pretty easy to navigate.  I can see though how cumbersome it is for the writers to upload, but it is still better than the paper version for many reasons.

Afternoon session

HCC reviewer

• if she's not familiar then she'll scan it
• asked if the system would flag if the applicant was applying early or late
• she had Ross and he did an introduction that explained how to go through his dossier
  • she thought it was helpful
  • well organized
• met face to face but the outer islands called in
• didn't paginate 
• for the chairs, would be nice to have confirmation that uploading the recommendation report and submitting was successful
  • TPRC process for manual and electronic was the same
  • survey monkey worked for her
• manual dossier was a lot longer

KCC

• easy to go through eligibility section
• extremely helpful with the dossier sections
• it was pleasant to go through the dossier
• she could jump around to sections and review as needed
  • very helpful because on manual she ends up re-reading a lot
• doesn't feel rushed

Christine 

• Just the whole process is so much easier than having to travel and share the binders with others
• could thoroughly review the application than the manual process
• easy to navigate
• document a process on what to do and how to do it as the TPRC chair

Windward CC

Old school people here but 1 is retiring

Doug didn't press the submit

• make button more noticeable

Eligibility was clear; nothing stood out as difficult; some didn't review because it was assumed that the coordinator vetted the info; used it to know what they applicant was going for and who it was

Dossier

• Mike had Mariko's and found it easier to review the single page doc than the sectioned off one
  • single page was easier to refer to than finding different tabs
  • 1) she had a TOC for overall structure and 2) hyperlinks to supporting docs were great
  • quicker than the paper binder because they didn't have to look up supporting docs manually; too much back and forth
  • business opportunity for faculty to contract out formatting their dossier if the faculty is not tech savy
• Kyle's dad said both electronic apps were easy to navigate
• Pua
  • make contact rehires electronic
    • 2 until tenure
  • meeting at a distance (she's from Kauai) made communication a bit harder
    • she likes face to face
    • had to do a lot of clarification
    • they used google chat which explains her pain
      • recommend google hangout in reviewer training
• 2nd session
  • it's the responsibility of the applicant to organize it well because they could put the wrong content in the wrong section
  • liked the single page (Doug and the other lady); one liked the sections because she's old school
  • philosophical discussion and observation: tendency to scroll through quickly made it odd mentally that they were scanning and skipping

pagination

• • pagination or lack thereof wasn't an issue when referring to it in discussions
  • people took screenshots and one person copy and pasted and shared it with the TPRC
    • Jeff Hunt found that shared C&P very helpful in discussion
  • japanese guy wanted pagination (his wasn't) to know how many pages he had left to review
    • his had the sections and he found it helpful

Frustrations

• • intial face to face and then phone conference with online voting via survey monkey
    • Mike could deduce who submitted the votes because the sample size is so small
    • building something into 
  • if not approved for promotion, committee would put comments as to how they could receive promotion
    • applicants wouldn't go in and read what the committee said
      • committee could have said "hey you could get promotion if you did this..."
    • some applicants wouldn't even provide supporting documentation when asked for it
    • contract renewals might open a pandoras box of issues vs a tenure review
  • find a better tool for the reviewers to do their voting

Likes

• • better than interfolio
  • review at own leisure
    • more thorough than a cursory view
    • not competing with the other members to veiw paper binder
    • no need to make appt and schedule 3 or 4 visits to review
  • saves time in terms of traveling

They have a dossier hui

• create repository of excellent and bad dossiers
• best practices for an electronic version
• being able to see it within the system

There were a couple of screening levels in selecting the 2017 applicants so we had good, tech savy faculty

• this is a big reason why the pilot was successful

Label at the Chancellor's recommendation

• possibly make the chancellor's report optional
• Doug says they provide as little detail as possible in case of a grievance
• real test of the system will be when we get a bad dossier that needs the chancellor to review carefully and write up a long report

suggestion

• keep track of where he left off

Leeward CC

• positive feedback from Susan Wood
  • In my opinion, in my role as a reviewer, the flexibility to review the document on my own time and as needed is the most valuable part of the electronic tenure/promotion process.
• Dossier
  • was better than Interfolio
    • links worked
    • dual screens helped reviewers
    • didn't timeout like interfolio
  • sections was better because reviewer didn't do it one sitting so she knew where she left off
  • area to take notes would have been helpful
    • litigation excuse
  • one applicant's documents (who's the applicant?)
    • linking close to a link vs directly on the link
    • could they upload powerpoint?
  • a convener requested that all people on the island come to their campus (Ross' applicant)
  • some did skype
  • survey monkey
    • straw vote but no second vote
    • committee didn't know the results of the straw vote until they met
  • google hangouts
    • works best with safari on mac than other browsers
    • instructions on how to setup meeting (via calendar and widget)
    • zoom is second option (free)
    • halawai (adobe connect)
    • polycom (struggle because you have to schedule it)
  • online was more refreshing; when you see a thick binder, you mentally are depressed because you don't want to go through it
  • infographs

Maui College

Period notifications to the committee if it's been sitting for too long

• sent x days after it goes to the committee
• monitoring 

Honolulu CC

Elibility

• easy for them
• they always look at it
• chancellors office looks at it

Ina

• organization was outstanding
• had supporting documents for everything
• 700% better than inerfolio
• very intuitive and easy to find
  • navigation was easy
• had page numbers on it because she noted it down for discussion

Started with zoom but then had difficulty with one member, then used skype

They liked the sections in the dossier

No one really cared regarding the voting because it was unanimous

• they voted out loud and it wasn't anonymous

Being able to copy and paste for the recommendation report would have been helpful

Email to the reviewers was clean and easy to understand

Couple did it on the tablet

• you can screenshot but we can't prevent that
• one had to install google drive on their device in order to view supporting documents