INSIDE THIS ISSUE  

Special Edition to Guide IEP Team Decisions

IEP Team Decision-Making for the Michigan Merit Examination

IEP Team Decisions for the English Language Proficiency Assessment

Accommodations Decisions for IEP Teams

NAEP News for Students with Disabilities

Tools for Determining Participation in State Assessment

What's the MATR?

Ethical Administration Practices

Medical Emergencies and the MEAS

MI-Access Assessment Development Update

Meeting NCLB Technology Literacy Requirements

Notes from the Contractor to District MI-Access Coordinators

Glossary

Important MI-Access Dates

Bookmark These Web Sites






P.O. Box 30008 Lansing, MI 48909 (517)335-0471

Mrs. Kathleen N. Straus (Detroit)
President

Mr. John C. Austin (Ann Arbor)
Vice President

Mrs. Carolyn L. Curtin (Evart)
Secretary

Mrs. Marianne Yared McGuire (Detroit)
Treasurer

Mrs. Nancy Danhof (East Lansing)
NASBE Delegate

Mrs. Elizabeth W. Bauer (Birmingham)
Board Member

Mr. Reginald M. Turner (Detroit)
Board Member

Mrs. Eileen Lappin Weiser (Ann Arbor)
Board Member


Ex Officio

The Honorable Jennifer M. Granholm
Governor

Michael P. Flanagan
Superintendent of Public Instruction

Funded by the Michigan Department of Education and the U.S. Education Department (USED).


If you have ideas, suggestions, or tips you would like to see included in The Assist, send them to

mi-access@tasa.com



DID YOU KNOW?

You can view the archived Spring 2006 Webcast at

mistreamnet.com



Excellent Resource Now Available!

Readers of The Assist may be interested in a new book, titled “Teaching Language Arts, Math, and Science to Students with Significant Cognitive Disabilities,” (copyright 2006, Paul H. Brookes Publishing Co., Inc.; Baltimore, MD; ISBN 1-55766-798-5). It was edited by Diane M. Browder, Ph.D. and Fred Spooner, Ph.D., and is available for purchase at online resources, such as Amazon.com.








































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































 

April 2006 (Volume 5, Number 3)


Special Edition to Guide IEP Team Decisions

Dear Readers,

Thank you for working so diligently to complete the spring assessment cycle for high school students. With such a challenging year due to changing assessment windows, exciting new high school initiatives, and new MI-Access assessments, it is heartening to see Michigan educators handle the flux with such professionalism.

Information for IEP Team Deliberation

Since many of you are currently engaged in the Individualized Education Program (IEP) planning process, we have dedicated this issue of The Assist mostly to providing you with information to help guide important decisions related to student participation in statewide assessment.

For example, two articles below provide valuable information on new or changing statewide assessments. The first article, titled “IEP Team Decision-Making for the Michigan Merit Examination,” describes the different components of the Michigan Merit Examination (MME) and important considerations for IEP Teams, such as what accommodations are available for students with disabilities and how to apply for them. The second article, titled “IEP Team Decisions for the English Language Proficiency Assessment,” describes the state’s newest operational assessment and provides information that should inform and enrich IEP discussions related to students who are English language learners and also have disabilities.

The article below, titled “Accommodations Decisions for IEP Teams,” should be helpful to teams as they select and evaluate assessment accommodations. It contains general information on the different types of accommodations as well as a list of questions to ask when selecting them for instruction and assessment. This article should be used in conjunction with the articles on the MME and ELPA when considering state-level assessment accommodations.

Furthermore, this issue of The Assist contains updated information on the National Assessment of Educational Progress (NAEP), including the content areas that will be assessed in the coming year and how many Michigan schools will be involved.

Finally, as in years past, we have included updated versions of three useful IEP Team decision-making tools: (1) an age-to-grade conversion chart, showing which MI-Access assessments should be taken by students who are ungraded in the state’s Single Record Student Database (SRSD), (2) a flow chart, which reminds IEP Teams of important factors to consider when selecting appropriate state assessments for their students, and (3) a checklist that accompanies the flow chart. Please feel free to copy these tools and use them during IEP Team deliberations.

Additional Assessment Information

In this issue of The Assist you will also find important information that is not directly related to IEP Team decisions but does impact the statewide assessment of students with disabilities. We have included articles on ethical assessment administration practices; medical emergencies that impact student involvement in the state’s assessment system; MATR, an important state resource to use when considering accommodations involving technology; technology literacy, an important component of No Child Left Behind (NCLB); and information from the MI-Access contractor—BETA/TASA—on upcoming timelines and materials handling procedures that are critical for ensuring smooth assessment and reporting cycles.

We conclude this issue of The Assist with an update on the progress of the MI-Access Assessment Plan Writing Teams (APWTs). They reconvened in January and March to continue their important work of developing extended content expectations, extended benchmarks, and assessment plans for the MI-Access assessments that will soon need to be operational. The article titled “MI-Access Assessment Development Update” provides details about these meetings. Many thanks are due to the APWT members for their diligent work on these challenging tasks.

Again, thank you for your continued efforts to stay current with the many changes surrounding state assessment. We at the Assessment for Students with Disabilities Program look forward to working with you throughout the remainder of the school year and thank you for your continuing support and professionalism.

Sincerely,

Peggy Dutcher
Coordinator, Assessment for Students with Disabilities Program

dutcherp@mi.gov




IEP Team Decision-Making for the Michigan Merit Examination

In spring 2007, the Michigan Merit Examination (MME) is scheduled to replace the current Michigan Educational Assessment Program (MEAP) High School Test (HST). The MME will be administered at that time if it is approved by the U.S. Education Department by November 1, 2006. If it is not approved, then the MEAP HST will be administered.

The timing of the approval process means that Individualized Education Program (IEP) Teams will need to consider appropriate assessment accommodations for both the MME and the MEAP HST for spring 2007. Beyond accommodations decisions, IEP Teams will need to decide if the MME is appropriate for their student, or if he or she should take MI-Access, which will continue to be available as an alternate. However, IEP Teams must keep in mind that the percentage of students who should participate in MI-Access will continue to be small, since those assessments are currently designed for students who have, or function as if they have, a cognitive impairment.

Components of the MME

The table below shows how the MME will be organized. There will be three assessment booklets and three corresponding answer folders. Schools will administer a particular set of assessments on each of three days.

• On the first day, ACT Plus Writing will be administered in five parts.

• On the second day, WorkKeys and Michigan mathematics will be

administered in three parts.

• Schools may choose to give the science and social studies

assessments in three back-to-back parts on any single day during

Days 2 through 7.

In addition to showing the MME’s organization, the table below provides the number of multiple choice (MC) items and prompts that will appear in each part of the assessment; the actual testing time; and the estimated time for administration, including check in, instruction, testing time, breaks, and collection of materials. All parts of each assessment will be timed and must be administered at the same time of day to all students in a given high school. These assessment organization parameters are designed to provide consistent and high levels of test security, increase test validity, and minimize the interruption of instruction.


Accommodations on the MME

There are two categories of accommodations for the MME. The first is ACT-approved accomodations, which can be used by a student and will result in (1) college reportable ACT scores, and (2) scores that can be used for making Adequate Yearly Progress (AYP) and Education YES! calculations, as well as for determining a student's eligiblity for the Michigan Merit Award. The second category is state-allowed accomodations, which will result in scores that count for the Michigan Merit Award, Education YES!, and No Child Left Behind (NCLB) AYP, but may not result in college reportable ACT scores. The differences come primarily from the fact that ACT is bound to provide accommodations under the Americans with Disabilities Act, while the state may permit additional accommodations under the Individuals with Disabilities Education Act (IDEA 2004) and Section 504 of the Rehabilitation Act. ACT must be more restrictive in order to ensure that college reportable scores are comparable nationwide.

MME Accommodations Summary Table

In order to facilitate provision of appropriate accommodations, the Office of Educational Assessment and Accountability (OEAA) has worked closely with ACT to develop an MME Accommodations Summary Table. This helpful document contains information on the MME component scores (for example, it lists all the pieces of the MME that contribute to the overall score for mathematics). The table also contains directions on how it should be read, details on requesting accommodations from ACT, a summary of the ACT review process, and information about local decisions for WorkKeys and the Michigan components of the examination. The table then lists all the accommodations from the State Board-approved OEAA Accommodations Summary Table (June 2005) for MEAP and MI-Access. Each accommodation contains comments from ACT and fields indicating whether or not it (1) is standard or nonstandard for Michigan components and MME scores; (2) may be requested for use on the ACT and WorkKeys components; and (3) will result in college reportable ACT scores. The table is available on the MME Web page at www.mi.gov/mme under the “What’s New” category.

Requesting Accommodations

If it is determined that a student needs accommodations on the MME, the accommodations must be officially requested. Each district is responsible for assigning a Test Accommodations Coordinator (TAC) to work with ACT to ensure that accommodations decisions and approvals are resolved before the assessment window opens. In the fall, ACT will send a mailing to each TAC (or school principal if no TAC has been designated) that includes two application forms customized for Michigan—one for ACT-approved accommodations and one for state-approved accommodations. The TAC is responsible for obtaining information from IEPs regarding which students need which types of accommodations and submitting supporting documentation for ACT-approved accommodations. The type of accommodation (ACT-approved vs. state-approved) is important because students receiving one type of approved accommodation will have different materials and be tested in a different room than students receiving the other.

The requesting process contains deadlines for application submission to ensure that ample time exists for each TAC to appeal any ACT-denied accommodations requests, and to apply for state-approved accommodations if the appeal is denied. ACT will send information along with any denial indicating why the request was turned down. Reasons may include insufficient documentation, no justification for the accommodation documented in the student’s IEP, or no history of the accommodation being used prior to planning for the ACT. Specific requirements for ACT-approved accommodations include documentation of initial diagnosis, reconfirmation of disability within the last three years, and a copy of the accommodations page from the student’s current IEP or 504 Plan.

Resources

More information on the MME can be found on the MME Web page at www.michigan.gov/mme, including the MME Accommodations Summary Table mentioned earlier and Michigan legislation documenting the replacement of the MEAP HST with the MME.

Another important resource is the MME Accommodations Briefing, which originally took place on March 6, 2006, and was archived at www.mistreamnet.org for continued viewing (click on “Archived Streams” in the left-hand column, then click on the MEAP and MI-Access icon under “Categories”). Videotapes or DVDs of the briefing may also be ordered from Brenda Hose at hoseb@resa.net or 734-334-1437.

Finally, another live videoconference is scheduled for April 27, 2006. It is called “The Michigan Merit Examination: Plans for Implementation in 2007,” and will cover a range of topics, including the assessment schedule, the MME content and scoring, required school personnel roles, applications for site approval, applications for accommodations for students with disabilities and English language learners; the benefits of ACT and WorkKeys to students and schools; and preparing schools and students for the exam. The videoconference will originate from Wayne RESA and will be distributed to a number of confirmed host sites. For more information on this upcoming event, go to www.mi.gov/mme and look under the “Assessment Accommodations” category.




IEP Team Decisions for the English Language Proficiency Assessment


Michigan’s English Language Proficiency Assessment (ELPA) is the latest component of the Michigan Educational Assessment System (MEAS) to achieve operational status. Administered for the first time in spring 2006, the purpose of the ELPA is to measure the English language proficiency levels of Michigan students who are learning English as a second language. The ELPA helps Michigan meet federal No Child Left Behind (NCLB) requirements for the annual assessment of students eligible as English language learners (ELLs), and is based on the English language proficiency standards that were adopted by the State Board of Education in April 2004. As noted in the Spring 2006 ELPA Coordinator Manual for Districts and Schools, the “long-term goal of the ELPA is to create uniform measures that support the teachers who help these students learn English so that our students can more effectively participate in academic courses taught in English.”

Eligibility for the ELPA

As outlined in the ELPA manual, all ELLs who are enrolled in grades K-12 at Michigan public schools, including charter schools, must be assessed with the ELPA each spring. This includes all ELLs who are eligible to receive services (including bilingual, English as a Second Language (ESL), and so forth), regardless of whether or not they are currently receiving them. All ELLs enrolled in private schools that receive Title III funding must also be assessed according to agreements with their local districts. Students are identified as eligible ELLs through the Home Language Survey, which is used to determine if the student’s native language is not English or if a language other than English is spoken in the home. If either condition is true, then the student’s English language proficiency is assessed. Students scoring below proficient are considered ELLs and are part of the pool that will take the ELPA.

ELPA and Students with Disabilities

Throughout the state there are students with disabilities who are also ELLs. Therefore, IEP Teams need to carefully consider whether students who are eligible for special education should also participate in the ELPA and, if so, what accommodations might be appropriate based on the student’s individual needs. Many IEP Teams have been utilizing the OEAA Assessment Accommodation Summary Table that was approved by the State Board in June 2005 to make accommodations decisions for MEAP and MI-Access. The table was recently expanded to include accommodations for ELPA, and was approved by the SBE on March 14, 2006. The table is important because the ELPA assesses the four English language arts areas of listening, reading, writing, and speaking. It is necessary to address each area separately for each student participating in the ELPA, since some accommodations are standard for one section but not for another. In particular, the area of speaking has not been assessed at the state level on any prior assessment and, thus, requires careful consideration.


Appendix A of the Spring 2006 ELPA Coordinator Manual for Districts and Schools contains the ELPA excerpt from the OEAA Assessment Accommodation Summary Table. The manual also has a section dedicated to accommodations that discusses audio versions; the use of scribes, tape recorders, and Braillewriters; the rules for using word processors; and other information that can help IEP Teams make informed decisions about what accommodations are available and how to provide them to students. As is the case for MEAP and MI-Access, nonstandard accommodations change the construct being tested. Therefore, their use will result in students being counted as not assessed, which may create accountability difficulties. For example, while ELPA participation rates are not used in the calucation of NCLB Adequate Yearly Progress (AYP), they are an important part of Michigan's NCLB Title III reporting requirements. The percent of students that must participate in the ELPA for Title III is the same percent that must participate for NCLB AYP calculations (a minimum of 95%) even though the percentages are used for different purposes. Michigan must, therefore, work to ensure that all ELLs are assessed each year, and while performance on the ELPA will not directly impact AYP calculations for a school or district, it is expected that as students make progress in learning English, the number of ELLs who score at the “proficient” level on MEAP and MI-Access should increase.


A few ELLs across the state have disabilities so significant that participation in the ELPA is not appropriate. IEP Teams should make this decision by ordering ELPA materials for each student in question, and reviewing them in light of the nature of the student’s disability. If the student can participate on some portions of the ELPA—for example, the speaking and listening portions—the assessment should be administered. If the student’s disability prevents him or her from participating on any section of the ELPA, that should be documented in the student’s IEP. To help the Michigan Department of Education (MDE) account for these students, ELPA coordinators need to send to the MDE (1) the most recent Present Level of Academic and Functional Performance (PLAFP), (2) the section of the student’s IEP where participation in state assessment (MEAP or MI-Access) is recorded, and (3) the student’s pre-printed ELPA answer document. All documentation should be sent to Marilyn Roberts, Michigan Department of Education, Office of Educational Assessment and Accountability, P.O. Box 30008, Lansing, MI 48909.

ELPA Resources

The MDE Web site includes an ELPA Web page, which can be found at www.michigan.gov/elpa. Among other information, this site contains:

• Rubrics for scoring the speaking component of the ELPA.

• Question & Answer documents from teleconferences held in January

and March 2006

The Spring 2006 ELPA Coordinator Manual for Districts and

Schools, which lists responsibilities for district and building

coordinators and assessment administrators; has information about

how to handle assessment materials before, during, and after the

assessment; contains Frequently Asked Questions, a glossary, and an

Accommodations Summary Table for ELPA; and provides information

about security and ethics.

• General information on state assessment of ELLs.

The archived ELPA teleconferences from January 20 and March 2, 2006, are available for viewing at www.mistreamnet.org (click on “Archived Streams” in the left-hand column, then click on the Special Education icon under “Categories”). Videotapes and DVDs of the March briefing can also be ordered from Brenda Hose at hoseb@resa.net or 734-334-1437. General comments and questions about the ELPA can be directed to the State ELPA Coordinator, Marilyn Roberts, at robertsm@michigan.gov.




Accommodations Decisions for IEP Teams


Making the best possible decisions regarding accommodations for individual students with disabilities is a critical but difficult task. As the statewide assessment system continues to evolve, Individualized Education Program (IEP) Teams are faced with the corresponding tasks of determining which assessment is most appropriate for a given student, as well as how to administer the assessment with accommodations that mirror those used in classroom instruction. The purpose of this article is to provide some reminders about what accommodations are, outline the goal of providing them, discuss some recommended questions that IEP Teams should ask during the accommodations selection process, and give some guidance on how the effectiveness of accommodations for individual students might be determined. Much of the information in this article comes from documents prepared by the Council of Chief State School Officers (CCSSO) Assessing Special Education Students (ASES) State Collaborative on Assessment and Student Standards (SCASS), in which Michigan is an active member.

Accommodations in General

Many special education service providers are exceedingly familiar with what accommodations are and how students should use them. However, since the use of accommodations in statewide assessment has potentially serious consequences for students and districts, it is useful to revisit their definitions and the thought processes behind providing them. It also is important to remember that (1) many assessment accommodations, such as having a mathematics assessment read to the student, need to be provided in a one-on-one situation; and (2) assessment accommodations should only be used if the student’s IEP indicates that they are appropriate for the student, and they reflect what the student routinely uses or how he or she routinely responds during instruction.

Fundamentally, accommodations are tools and procedures in the areas of presentation, response, timing/scheduling, and setting that provide access to instruction and assessment for students with disabilities. It is important to bear in mind that accommodations minimize the effects of a student’s disability and do not reduce learning expectations. They provide a means for accessing the content and instruction being provided. The four general categories into which nearly all accommodations fit are described below.

Presentation accommodations allow students to access

information in ways that do not require them to visually read standard

print. These alternate modes of access are visual, tactile, auditory, and

multi-sensory.

Response accommodations allow students to complete

assignments, assessments, and activities in different ways or to solve or organize problems using some type of assistive device or

organizer.

Timing/Scheduling accommodations increase the time allowed to

complete an assessment or assignment and may also change the way the time is organized.

Setting accommodations change the location in which an

assessment or assignment is given or the conditions of the

assessment setting.

In contrast to providing access to content through accommodations, modifications are alterations in learning or assessment expectations. Examples of modifications include requiring a student to learn less material, or revising assessments or assignments to make them easier. Since modifications, or nonstandard accommodations, change either the scope or content being measured, they should be considered carefully for use on statewide assessments. If a student uses a modification or nonstandard accommodation when administered any of Michigan's state-level assessments, the student will count as not participating in the assessment, which will impact No Child Left Behind (NCLB) participation rates.

Goals of Accommodations Provision

The primary driving force behind providing accommodations is access. In the case of students with disabilities, that means access to grade-level content to the greatest extent possible based on the nature of the disability. To make this possible, every IEP Team member must be familiar with the state’s content standards and local curricula. In order to effectively reach decisions that will have the maximum beneficial impact for students, collaboration between general and special educators is paramount. Accommodations must be utilized consistently and appropriately if students are to have access to the educational content they deserve. As illustrated below, the use of accommodations should be linked across instruction and assessments, as well as constitute a fundamental part of each student’s ecology.


Selecting Accommodations

Making accommodations a part of a student’s academic experience is an important decision that should not be taken lightly. IEP Teams need to consider several issues before making any decisions. For example, IEP Teams should never check off every accommodation available in the Office of Educational Assessment and Accountability (OEAA) Assessment Accommodation Summary Table or employ as many standard accommodations as possible on an assessment hoping that “something” will work. Instead, they should work to match a student’s particular needs with the accommodations that can best address them.

Following are four different lists of questions, issues, and actions that IEP Teams may want to use to guide their discussions about selecting accommodations for students with newly identified disabilities or reviewing those that are already in use.

Guiding Questions to Consider

• What are the student’s learning strengths and needs?

• How do the student’s needs affect the achievement of grade level

content standards?

• What specialized instruction (e.g., learning strategies, organizational

skills, and/or reading skills) does the student need to achieve grade

level content standards?

• What accommodations will increase the student’s access to

instruction and assessment?

Reviewing Current Accommodations

• What accommodations are currently used by the student in the

classroom and on assessments?

• What were the assessment and assignment results when

accommodations were used and not used?

• Were any effective combinations of accommodations used?

• Were there any difficulties related to the accommodations used?

• What was the student’s perception of how well the accommodations

“worked?”

• What were the perceptions of the parents, teachers, and specialists

about how well the accommodations “worked?”

Considering New Accommodations

• What are the student’s access needs and what possible

accommodations could he/she try? (Make a list.)

• Of the accommodations on the list, consider

the student’s willingness to learn to use the accommodations,

the opportunities that are available to learn how to use them in

classroom settings, and

the conditions for using them on state assessments.

Planning for the Use of New Accommodations

• How will the student learn to use each new accommodation? (Develop a plan.)

• Is there plenty of time for the student to learn to use the instructional

and assessment accommodations before testing?

• How can we evaluate and improve the use of accommodations on an

ongoing basis?

Involving students in the process of selecting their own accommodations can greatly increase the chance that they will be used and that the student will benefit from their use. In addition, it can help students start seeing accommodations as part of their everyday life, and lead to discussions about how they might be used outside of school.

The OEAA Assessment Accommodation Summary Table

While the questions and issues listed above are useful in guiding discussions about accommodations, it is also helpful to have a list of commonly used accommodations as a reminder of what can be considered for statewide assessment. The purpose of the State Board of Education (SBE)-approved OEAA Assessment Accommodation Summary Table is to provide educators, parents, and other interested parties with a summary of the standard (S) and nonstandard (NS) accommodations for each state assessment included in the SBE-adopted Michigan Educational Assessment System (MEAS). The MEAS includes the Michigan Educational Assessment Program (MEAP), MI-Access (Michigan’s Alternate Assessment Program), and the English Language Proficiency Assessment (ELPA). In addition to the accommodations for the assessments included in the MEAS, the table includes permitted (P) and not permitted (NP) assessment accommodations for the National Assessment of Educational Progress (NAEP). The accommodations available for the spring 2007 administration of the Michigan Merit Examination (MME) are available in a separate table, which is discussed in detail in the article called “IEP Team Decision-Making for the Michigan Merit Examination.”

The summary table should not to be used as a checklist for determining what assessment accommodations should be used for a student, but to make sure that appropriate ones are selected and that the consequences of using nonstandard accommodations are considered. The introductory pages of the summary table should be read carefully, as they outline the impact of using standard or nonstandard accommodations on Adequate Yearly Progress (AYP) calculations and student eligibility for the Michigan Merit Award. The summary table is posted on the MI-Access Web page at www.mi.gov/mi-access under the “Resources” category.

Evaluating the Effectiveness of Accommodations

Following are some additional questions that IEP Teams may find useful when deliberating about accommodations. Answering these questions together—as teams are preparing to hold IEPs or following the administration of classroom or statewide assessments—can help ensure that each student is being allowed the best possible access to content that is appropriate for him or her.

• What accommodations are used by the student in the classroom and

on assessments?

• What are the results of classroom assignments and assessments

when accommodations are not used?

• What is the student’s perception of how well each accommodation

“worked?”

• What seem to be effective “combinations” of accommodations?

• What are the difficulties encountered in the use of accommodations for a student?

• What are the perceptions of teachers and others about how the

accommodation appears to be “working?”




NAEP News for Students with Disabilities



The National Assessment of Educational Progress (NAEP) is administered to a variety of schools and covers different content areas every year. The goal of the NAEP is to gather assessment data from a representative sample of all students in the nation, including students with disabilities (see nces.ed.gov/nationsreportcard/about/

inclusion.asp). NAEP administration is required by the Michigan School Code, but it is voluntary for students—that is, they may opt out by going through a special notification process. For students with Individualized Education Programs (IEPs) who are selected for NAEP testing, it is assumed that they will participate in the sample unless (1) they and/or their parents decide to have them opt out, (2) their IEP indicates they will not take part, or (3) the accommodations they require do not match the NAEP-allowed accommodations (see the OEAA Assessment Accommodation Summary Table at www.mi.gov/mi-access.) This year, there were some schools that assumed it was not a NAEP testing year and, therefore, wrote into IEPs that students would not have to participate in the NAEP. This caused several inadvertent problems later in obtaining permission from parents. To prevent these problems in the future, please review the accommodation procedures and ensure that parents and school staff understand the NAEP test cycles and inclusion practices.

Each year, NAEP draws its sample of schools and students from data submitted by the Center for Educational Performance and Information (CEPI). The NAEP testing cycle for the 2005/2006 school year ran from February to mid-March. This was a national sample year that involved the participation of 26 Michigan schools. The subjects tested were U.S. history (grades 4, 8, 12); civics (grades 4, 8, 12); and economics (grade 12). Next year is a Michigan sample so the sampling frame will be larger, covering 400 to 500 schools. The subjects for 2007 testing are reading (grades 4 and 8), mathematics (grades 4 and 8), and writing (grades 8 and 12). In addition, a small number of schools will be sampled for a twelfth-grade Participation and Engagement Study, which will assess whether students do better in the fall or winter on mathematics and reading assessments.

For more information on the NAEP, please go to www.michigan.gov/naep or contact Paul Stemmer, Michigan’s NAEP coordinator, at stemmerp@michigan.gov.




Tools for Determining Participation in State Assessment

Each year in the April issue of The Assist, we provide three updated tools for Individualized Education Program (IEP) Teams to use when selecting the state assessments in which their students will participate:


1. a table, which will be helpful in determining which MI-Access

assessment a student should take if he or she is ungraded in the

state's Single Record Student Database (SRSD);

2. a flow chart, which outlines the decision-making process, and

3. an accompanying checklist, which reminds IEP Teams of the

important factors to consider when selecting state-level assessments.

Please feel free to copy these three tools and use them during IEP Team deliberations.










What’s the MATR?

The need for appropriate accommodations to help students access instruction and assessment continues to increase dramatically. One way to help students is through assistive technology (AT). Unfortunately, many people are not familiar with AT and how it can be used in the classroom and during assessment.

Michigan’s Assistive Technology Resource (MATR) is available to assist districts, schools, and IEP Team members with expanding their knowledge of AT. The overall purpose of MATR is to provide information services, support materials, technical assistance, and training to local and intermediate school districts in Michigan to increase their capacity to address the needs of students with disabilities through AT. For more information or inquiries, contact MATR at Michigan’s Assistive Technology Resource, 1023 South U.S. 27, St. Johns, MI 48879, 800-274-7426, e-mail: matr@edzone.net.

Below is information about the MATR Summer Institute, an excellent opportunity to learn more about this important organization and their work.




Reaching and Teaching Every Student:

Universal Design for Learning

During this full day seminar, Skip Stahl from the Center for Applied Special Technology (CAST) will present an overview of Universal Design for Learning (UDL) in K-12 instructional settings.

Drawing from the CAST book, Teaching Every Student in the Digital Age, ASCD, 2002 (which is available online at www.cast.org/tes), this session will review the neurological foundation of UDL, emphasizing learner differences within the framework of neural networks. Building on a research-based understanding of brain function, the session will link what is known about how the learning brain works to cognitive psychology and, ultimately, to the tools of learning—the curriculum.

From this conceptual framework, the session will present a range of practical technology-based strategies and universally-designed learning supports for struggling students in grades K-12. In addition, the session will explore the resources available on CAST’s Teaching Every Student Web site. A guided tour of the links, sections, and sharing opportunities will be provided. Through an investigation of technologies that support UDL in the general education classroom, session attendees will gain a broader understanding of the ways in which digital media and tools can enhance the possibility of success for students with diverse learning needs. For additional information, please visit our website at www.cenmi.org/matr.




Michigan’s Assistive Technology Resource is an IDEA Mandated Activities Project awarded by the

Michigan Department of Education, Office of Special Education and Early Intervention Services.




Ethical Administration Practices

One purpose of state assessment is to accurately measure and report student achievement or what students know and are able to do in relation to Michigan’s content standards, specifically in the areas of English language arts, mathematics, science, and social studies. In order to achieve accurate data and ensure equity for all students, however, the assessments must be administered in a standardized manner. To that end, the Office of Educational Assessment and Accountability (OEAA) has developed procedures for districts and schools to follow when administering state-level assessments. Failure to follow these procedures could result in unethical practices with serious consequences for the students, schools, districts, and staff.

As noted in the MI-Access Fall 2005 and Spring 2006 Coordinator and Assessment Administrator Manuals, district and school staff can assist with ensuring accuracy and equity by doing five things: (1) maintaining the security of ALL assessment materials, (2) administering the assessments in a manner consistent with established assessment procedures, (3) establishing assessment conditions that prevent opportunities for students to engage in irregular behaviors, (4) reviewing student information on completed scan documents, and (5) following the communication procedures established for asking and answering questions. The Michigan Educational Assessment Program (MEAP) and English Language Proficiency Assessment (ELPA) administration manuals have similar information related to security.

In August 2005, the State Board of Education (SBE) approved a document called Professional Assessment and Accountability Practices for Educators (commonly referred to as “the ethics document”) to help meet the goal of ethical, standardized administration. The document was developed by the OEAA using professional guidelines that outline best practices in assessment and reporting of results. The ethics document has sections describing the roles and responsibilities of those involved with assessment administration, general issues of security and preparation, and what to do if an irregularity or unethical practice is suspected. It also has flow charts depicting the investigation process and a glossary. Everyone involved with the Michigan Educational Assessment System (MEAS) should read this document and understand the “ground rules” for administering state assessments. The ethics document is available on the OEAA Web page at www.mi.gov/oeaa.


Two examples of ethical irregularities are presented below as well as some helpful tips on how to avoid them.

Example 1: A student does not follow the standardized administration procedures. For example, he or she copies another student's answers or communicates or collaborates with another student during

assessment administration. (Communication includes written,

electronic, verbal, or gestured forms of communication.) To avoid this

situation, assessment administrators need to make it clear to students that all types of communication with other students during the

assessment are prohibited. In the case where students may respond

orally to questions, the IEP Team should consider having the

assessment administered individually to the student in a setting where oral comments will not be heard by other students.

Example 2: Teachers or proctors do not follow the standardized

administration procedures. For example, an assessment

administrator reads a portion of the assessment aloud, when reading

aloud is prohibited. This violation will cause scores to be invalidated,

resulting in some schools not making Adequate Yearly Progress

(AYP). To avoid this situation, districts must make certain that all

MEAP and MI-Access coordinators and assessment administrators

are thoroughly aware of administrative procedures and provide

substantive training to all proctors.

When problems occur or are discovered, an OEAA staff member must be contacted immediately. The table below shows who to contact depending on the assessment that was being administered when the violation or unethical practice took place.



Depending on the particular situation, the OEAA can sometimes rectify the situation or approve the use of emergency tests, which must be specially ordered at a cost to the district of $50 per student per content area. Getting approval to administer emergency tests will prevent a school from possibly not making AYP as a result of invalid scores, but the expense is yet another reason why it is wise to have high quality, careful training and close monitoring of coordinators, assessment administrators, accommodations providers, and proctors.




Medical Emergencies and the MEAS


It is the policy of the Michigan State Board of Education that all students participate in the state assessment system. Similarly, federal law (IDEA 2004 and No Child Left Behind 2001) requires that all students, including students with disabilities, be assessed through each state’s assessment system. However, obtaining full participation in state assessment is extraordinarily difficult due, in large part, to such uncontrollable circumstances as medical emergencies. That is why when calculating No Child Left Behind (NCLB) Adequate Yearly Progress (AYP), the minimum participation rate is 95 percent of enrolled students at each grade level.

If a student in a school or district has a medical emergency that precludes him or her from participating in state assessment, an exception can be granted by the Office of Educational Assessment and Accountability (OEAA). Exceptions due to medical emergencies are handled on a case-by-case basis, and evidence must be provided by schools either during the assessment period or the report card appeals process. The preferred method of notifying the OEAA is through an e-mail message sent to aypcontactus@michigan.gov. Please include the following information in your correspondence:

• Name of Student

• Grade level

• Name of School and District

• Date(s) of the emergency

• Medical condition or diagnosis

• Date(s) the assessment is scheduled to be administered at the school

Hospitalization and debilitating illness are two examples of “approved” medical emergencies, whereas non-limiting illnesses and pregnancy are examples of situations that would not be approved.


MI-Access Assessment Development Update


Throughout the 2005/2006 school year, the MI-Access team has continued to engage in the assessment development process first described in the August 2005 issue of The Assist. In January and March, the Assessment Plan Writing Teams (APWTs) met to continue their work on developing science alternate assessments for Functional Independence (FI), Supported Independence (SI), and Participation (P). In February, Content Advisory Committees (CACs) and Sensitivity Review Committees (SRCs) assembled to review hundreds of English language arts and mathematics items to ensure they are appropriate for students who take the Functional Independence assessments. Following is additional information presented by assessment status—either development or maintenance. "Development" refers to the production of new assessment components, and "maintenance" refers to the continual work needed to replenish items for existing assessments.

Development

MI-Access Science

Draft sets of extended benchmarks (EBs) have been completed for FI, SI, and P. The science benchmarks for all three MI-Access populations are organized by grade cluster (elementary, middle school, and high school) to correspond with the three grades in which science is assessed at the state level (grades 5, 8, and 11). The APWTs reconvened in January and March to finish drafting the EBs and to look across the extensions for all three MI-Access populations to ensure consistency and appropriateness of content.

In January and at the beginning of the March meeting, the APWT was divided into groups to consider the science benchmark strands (Constructing New Scientific Knowledge, Reflecting on Scientific Knowledge, Earth Science, Physical Science, and Life Science) in light of the content that might be appropriate for each of the MI-Access populations. After finishing this difficult task, the groups were reorganized to look at the selected content for a single student population. For example, one APWT group concentrated on the content chosen for SI and made sure it was applied consistently across the grade clusters and that the extended benchmarks at each level were appropriate for all five science strands.

The second portion of the March meeting was spent discussing how to write appropriate items or observable tasks for each MI-Access population. The teams considered multiple-choice and observation formats and made their preliminary recommendations to the Michigan Department of Education (MDE) on format. The teams also discussed important topics related to the proposed assessment plan for each population, such as philosophy and beliefs about the populations being assessed, how much time each assessment should take, and how many items should be included. Finally, the teams made recommendations for item specifications, which will guide item writers and make sure that the APWT’s recommendations for item length, artwork, and language are consistently applied. All of this work will culminate in a proposed science assessment plan, which is in the process of being developed and will be made available for public review and comment by Fall 2006.

MI-Access English Language Arts and Mathematics

Draft sets of extended grade level content expectations (EGLCEs) for English language arts (ELA) and mathematics have now been completed for the P and SI student populations and are nearly ready for field review. When available, they will be distributed in a format where reviewers can see the progression from grade level content expectation (GLCE) to FI EGLCE to SI EGLCE to P EGLCE. The purpose of presenting them in this manner is to enable reviewers to assist the P/SI APWT in making sure that all the content expectations addressed are appropriate for each population. Please keep in mind that there is not always a one-to-one correspondence. For example there might be five GLCEs in mathematics covering “count, write, and order whole numbers,” but only two FI EGLCEs and one SI EGLCE that address this content.

Below is an example of how the information on the EGLCEs will be presented. It shows, for illustrative purposes only, how a GLCE from third grade mathematics was reduced in breadth, depth, and complexity by the APWTs to be appropriate for each MI-Access population. No Child Left Behind (NCLB) offers a choice between grade-specific or grade-cluster alternate achievement standards; grade-specific standards were chosen for FI (e.g. grade 3) and grade-cluster standards were chosen for P/SI (e.g. elementary).



MI-Access Assessment Development Status

The table below shows where the MI-Access program is currently and what assessments are on the horizon. It is organized by the grade level at which the assessments are or will be administered. The column labeled “Currently Administered” lists the MI-Access assessments that are operational at this time; the column labeled “Under Development” includes the English language arts, mathematics, and science assessments that are in the process of being created; and the column labeled “Future Development” indicates the assessments that will need to be constructed at some point down the road.



Maintenance

The FI Sensitivity Review Committee (SRC) and Content Advisory Committee (CAC) meetings held in February had two purposes—to (1) review additional FI mathematics and English language arts items; and (2) review the embedded field test items from the FI fall 2005 assessments and their corresponding data. The SRC facilitators ensured that committee members understood the sensitivity and biasing elements that were crucial to consider for each item, and the CAC facilitators ensured that each item was reviewed for content appropriateness and accuracy. The CAC members also confirmed that each item measured the EGLCE or EB to which it was written.


The SRCs were broken down into two groups by content area—English language arts (ELA) and mathematics. The CACs were broken down into eight groups divided by grade/grade-clusters and by content area (mathematics grades 3-4, 5-6, 7-8, and 11, and ELA grades 3, 4-5, 6-8, and 11.) All groups reviewed the new items for their respective grade(s) and content area. However, only certain groups reviewed the fall 2005 field-test items and data. These groups included both SRCs; the mathematics grades 3-4 CAC; and the ELA grades 3, 4-5, and 6-8 CACs.

Based on their review, SRC and CAC members could recommend that each new item be accepted, revised, or rejected. In addition, the SRC and CAC members that reviewed the fall 2005 FI mathematics and ELA field-test items and data could recommend that an item either be accepted or rejected. If significant revisions to an item were recommended, the item was deemed ineligible for the fall 2006 assessments. These revised items will have to be embedded as a field-test item a second time or dropped.

Reviewed items that the committees recommended be accepted by the MDE are eligible for inclusion as embedded field-test items in the fall 2006 grades 3-8 assessments or spring 2007 grade 11 assessment. The process of embedding items is important and cost-effective as it allows student performance to be assessed without stand-alone piloting or field tests. The fall 2005 field-test items that the committee recommended to accept are also eligible for inclusion in the fall 2006 grades 3-8 operational assessments.

Look for additional assessment development updates in future issues of The Assist.





Meeting NCLB Technology Literacy Requirements

The portion of the No Child Left Behind (NCLB) Act known as the “Enhancing Education Through Technology Act of 2001” (E2T2) has one primary goal—to assist every student in crossing the digital divide by ensuring that they are technologically literate by the time they finish the eighth grade, regardless of their race, ethnicity, gender, family income, geographic location, or disability.

To meet this requirement, the Michigan Office of Grants and Education Technology is asking districts how they plan to determine technology literacy, which is defined by the State Education Technology Directors Association as “the ability to responsibly use appropriate technology to communicate; solve problems; and access, manage, integrate, evaluate, and create information to improve learning in all subject areas and to acquire lifelong knowledge and skills in the 21st century.”

Districts will be required to certify in the Michigan Electronic Grant System (MEGS) that they are working toward the goal of having every 8th grader be technologically literate by reporting the percentage of students who meet the above definition. The report must be submitted by June 30, 2006. The Michigan Department of Education (MDE) is developing technology standards that will be based on those created by the International Society for Technology in Education (ISTE). The standards should form the basis of what constitutes a technologically literate student. The standards can be reviewed at www.techplan.org.


Following are some commonly asked questions that were excerpted from an MDE fact sheet on eighth grade technology literacy. The answers may help begin to address some of the questions and concerns surrounding the standards, administration, and reporting requirements of the E2T2. The full FAQ is available at www.techplan.org/FAQsTechLiteracyRequirement.pdf.



Michigan Eighth Grade Technology Literacy

Questions and Answers Excerpted from the MDE’s FAQ

Q1: Will the state require schools to administer a formal assessment to determine if their eighth-grade students are technologically literate?

A1: The MDE has no plan to require a school district to administer any type of assessment to test for technology literacy skills. The local district will make the determination of how student technology literacy is determined.

Q2: Will a school district be required to report the progress of their eighth-grade students’ achievement of technology literacy by the end of the 2005/2006 school year?


A2: Yes. You are required to certify in MEGS that you are working toward the goal that every eighth grader is technologically literate and report the percentage of your students who are technologically literate.

Q3: Will the MDE provide districts with a definition of a “technologically literate student?”

A3: It is recommended that districts use the definition crafted by the State Education Technology Directors Association (SETDA), which states that “technology literacy is the ability to responsibly use appropriate technology to communicate; solve problems; and access, manage, integrate, evaluate, and create information to improve learning in all subject areas and to acquire lifelong knowledge and skills in the 21st century.”

Q4: Will the MDE provide districts with any guidelines or suggestions for measuring student technology literacy?

A4: Yes. We are aware that there are various methods that can be used to demonstrate technology literacy. Some suggestions and related links follow.

Knowledge based assessments:

•Summative or formative tests

•ISTE/Microsoft Assessment (formative)

•ISTE/ICDL/Vantage Learning online assessment

•NCRTEC online assessment (currently under development, ready in late spring 2005)

•Completion of Computer Literacy Class

Performance based assessments:

•Teacher observations

•Checklists (Example checklist from NCRTEC)

•Artifacts

Portfolio based assessments:

•Student reflective narratives

•Collection of artifacts

•Digital or paper portfolios

For more information on assessment, please review the REMC Tool Kit at www.remctoolkit.org/wiki/index.php/NCLB_Technology_Literacy Requirements.

Q5: Will the MDE require some type of formal authentication?

A5: No. No formal MDE authentication procedure is envisioned at this time. Federal requirements, if imposed, may require authentication in the future.


Q6: What documentation will be expected from the MDE?


A6: It will be up to the district to determine the most efficient method for documenting the technology literacy of their students. Documentation, at a minimum, should be able to disaggregate data to the individual student level.

Q7: What are the consequences if a district does not certify in MEGS?

A7: There is no indication at this time that failure to certify the technology literacy levels of eighth-grade students will result in a reduction or loss of federal grant funding for the district.

Q8: Is there a Web site that we can reference for additional information regarding technology literacy?

A8: Yes. Please visit techplan.org/literacy.html for links to a wide variety of sites relating to technology literacy.

Q9: To whom should we direct questions regarding eighth grade student technology literacy?

A9: Please contact either Barbara Fardell (fardellb@michigan.gov) or Ron Faulds (fauldsr@michigan.gov)




NOTES FROM THE CONTRACTOR TO DISTRICT MI-ACCESS COORDINATORS



Return of Spring 2006 MI-Access Grade 11 Assessment Materials

Greetings from BETA/TASA, the MI-Access assessment contractor. We hope the administration of the Spring 2006 Grade 11 MI-Access assessments went smoothly in your district. As you know, the assessment window closed April 14, 2006. District MI-Access Coordinators needed to ship all used and unused assessment materials (excluding rulers and Spring 2006 MI-Access Coordinator and Assessment Administrator Manuals) back to BETA/TASA NO LATER THAN April 21, 2006. This date was critical to ensure that materials were scanned, scored, and reported before the State Board of Education met to approve recommended student performance standards, which will be set in early May. Remember, if your district’s results were not postmarked by April 21, 2006, your students will count as not participating in your No Child Left Behind (NCLB) participation rates and Adequate Yearly Progress (AYP) calculations.

Early Return Districts

Thank you to the many districts who agreed to participate in the MI-Access Early Return Program by returning MI-Access assessment materials before April 12, 2006. Returning materials early ensures your district’s results will be included in standard-setting data and also enables you to view your Tested Roster earlier than others. We encourage all districts to take advantage of this opportunity in the future, if possible.

Spring 2006 MI-Access Grade 11 Tested Roster and Demographic Updates

Once BETA/TASA has scanned and scored the MI-Access assessments, District Coordinators will have an opportunity to review their Tested Rosters at the OEAA Secure Site (www.michigan.gov/oeaa-secure) to verify that (1) all students who were assessed are accounted for, and (2) each student’s demographic information is correct. This is necessary to produce accurate reports and data for calculating NCLB participation rates and Adequate Yearly Progress.


The statewide verification of Tested Rosters began on April 21 and will continue through May 12, 2006. However, each district’s Tested Roster information will only be posted for a two-week window during that period. This window is determined by the date on which a district’s assessment materials were received and processed by the contractor. Each District MI-Access Coordinator will receive an e-mail notice indicating that his or her district’s data have been posted to the OEAA Secure Site. To ensure that you receive this important notice, please verify that your District MI-Access Coordinator information is updated and accurate in the MI-Access Online System.

During this two-week window, District MI-Access Coordinators will review Tested Rosters in order to verify that all students are accounted for and to submit research requests for any apparent discrepancies. District MI-Access Coordinators may review Tested Rosters ONLY during the two-week window indicated in the e-mail notice from Peggy Dutcher and BETA/TASA. No extensions will be granted.

Please note that while demographic updates may be made at any time, the Tested Roster window is the final opportunity for districts to update demographic information prior to printing MI-Access District, School, Class, and Individual Student Reports as well as calculating Adequate Yearly Progress and participation rates. District Coordinators may want to start reviewing and updating student demographic information prior to the posting of their Tested Roster in order to reduce their workload during the designated Tested Roster update period.

Fall 2006 MI-Access Teacher/Student Counts

BETA/TASA will soon begin collecting Fall 2006 MI-Access Grades 3-8 teacher and student counts. The MI-Access Online System will be used to collect counts beginning on May 8, 2006. We realize that it is difficult for districts to anticipate their exact needs for the following school year. However, it is very important for BETA/TASA to receive counts that are as accurate as possible in order to produce the proper amount of MI-Access assessment materials over the summer. We hope you can assist us by working with schools and teachers to submit these important counts before June 14, 2006. Please note that if there are any changes to the aforementioned dates, you will be notified via e-mail. Also note that if you do NOT submit counts, you will NOT receive assessment materials next fall.

Fall 2006 MI-Access Pre-Identification

The pre-identification (Pre-ID) process for the Fall 2006 MI-Access Grades 3-8 assessment window is also quickly approaching. Once again, MI-Access students will be pre-identified entirely at the OEAA Secure Site. We encourage all District MI-Access and MEAP Coordinators to work together to assign students to the Fall 2006 MI-Access test cycle and to designate the proper MI-Access assessment for each student.

The Pre-ID process for MI-Access students is expected to begin on May 19, 2006. BETA/TASA will provide preprinted observation sheets or answer documents for students who are pre-identified by September 15, 2006 for the MI-Access Participation, Supported Independence, or Functional Independence Mathematics assessments. Students who are pre-identified for the Functional Independence English Language Arts (ELA) assessment before September 15, 2006, will receive a preprinted ELA barcode label to affix to the ELA answer document. If students are pre-identified in the MI-Access test cycle, but are not assigned to a particular MI-Access assessment type, only generic MI-Access preprinted barcode labels will be provided. After September 15, 2006, districts and schools will need to use the OEAA Secure Site to locally print barcode labels.






Assessment Item: A statement, question, exercise, or task in an assessment for which the test taker is to select or construct a response or perform a task. (Standards for Educational & Psychological Testing, 1999.)

Assessment Plan: Much like a builder’s blueprint, an assessment plan guides how an assessment is built or developed. It includes detailed information on (1) the assumptions underlying the assessment; (2) the populations and subject areas assessed; (3) the number of assessment items and their formats; (4) prototype items to guide item writers; and (5) other information clarifying how and why the assessment should be developed.

Professional Assessment and Accountability Practices for Educators: “The Ethics Document,” prepared by the Office of Educational Assessment and Accountability (OEAA) to guide appropriate assessment practices.

Standard Assessment Accommodation: The goal of an assessment accommodation is to minimize the impact of a student’s disability on his/her performance on an assessment. The assessment accommodation is considered “standard” if it does not change what a specific assessment is measuring. The score received by a student using a standard assessment accommodation will count when calculating No Child Left Behind participation rates. A “nonstandard” assessment accommodation—which does change what a specific assessment is measuring—results in an invalid score.

Technology Literacy: Technology literacy is the ability to responsibly use appropriate technology to communicate; solve problems; and access, manage, integrate, evaluate, and create information to improve learning in all subject areas and to acquire lifelong knowledge and skills in the 21st century.


Important MI-Access Dates


MI-Access Assessment Materials Shipped Back to BETA/TASA

No later than April 21, 2006


Tested Roster Review

April 21-May 12, 2006

MI-Access Grade 11 Functional Independence Standard Setting

May 4 & 5, 2006

Fall 2006 Teacher/Student Counts (MI-Access Online System)

May 8-June 13, 2006

Fall 2006 Pre-ID Begins (OEAA Secure Site)

May 19, 2006


MI-Access Grade 11 Functional Independence English Language Arts Rangefinding

June 12 & 13, 2006


2006 OEAA Fall Conferences

Nov 28 & 29Marquette, Northern Michigan University

Nov 30 & Dec 1Grayling, Holiday Inn

Dec 4 & 5Grand Rapids, Crowne Plaza on 28th Street

Dec 7 & 8Novi, Sheraton Detroit Novi

Dec 11 & 12Lansing, Lansing Center

Dec 13 & 14Sterling Heights, Sterling Inn




Bookmark These Web Sites



www.mi.gov/mi-access

MI-Access

www.mi-access.info

MI-Access Information Center

www.mi.gov/oeaa

Office of Educational Assessment and Accountability

www.mi.gov/meap

Michigan Educational Assessment Program (MEAP)

www.mi.gov/mme

Michigan Merit Examination

www.mi.gov/elpa

Michigan English Language Proficiency Assessment

www.mi.gov/naep

National Assessment of Educational Progress

www.cenmi.org/matr

Michigan Assistive Technology Resource

www.nochildleftbehind.gov

U.S. Education Department

www.nasdse.org

National Association of State Directors of Special

Education


This online newsletter related to the assessment of students with disabilities is available to local and intermediate superintendents, directors of special education, MI-Access Coordinators, MEAP Coordinators, SEAC, Special Education monitors, MDE staff, school principals, Parent Advisory Committees, and institutes of higher education.