Release Notes v5.1

This document provides a description of the technical enhancements to Generate version 5.1 released in October 2022.

RELEASE OVERVIEW

Generate version 5.1 continues to leverage a new data migration approach first introduced in version 4.2 of Generate in which data are migrated from Staging tables directly to the RDS Fact and Dimension tables. For SEAs using the Generate user interface to migrate data, this new migration method requires no changes to the loading of the staging tables and requires no changes to how you currently migrate data. Please continue to run the ODS Migration, RDS Migration, and Reports Migration to create submission files.

For SEAs that use manual migration scripts that reference the individual stored procedure names that are saved locally, and that DO NOT use the wrapper scripts for that file group (Child Count, Discipline, Exiting, Personnel, or Directory), for example (App.Wrapper_Migrate_ChildCount_to_RDS), you will need to update them to call the wrapper script because the manual migration scripts will no longer be valid. The wrapper script handles the new process, and that is the only change needed.

The benefits of this change include:

• Simplifying the steps in the migration process

• Less debugging as data is migrated

• The migrations run faster

State ETL changes:

There was a stored procedure named Staging.Migrate_Data_Validation that was used in earlier versions of Generate. We will be implementing a new migration validation procedure, so we removed this deprecated stored procedure. You will just need to verify that your Source to Staging ETL code does not call this stored procedure. If so, remove the call.

GENERATE ENHANCEMENTS

The following EDFacts reports were updated in this release.

Type of Impact:

• Data – the changes will improve data quality and completeness

• User Interface – the changes impact the Generate User Interface and/or migration process

• Source to Staging ETL – the changes may require modifications to the SEA’s Source to Staging ETL

• Performance – the changes improve the performance of a data migration

• Migration – the changes impact a data migration process

• Submission Files – the changes may impact submission file(s)

• Database – changes to the Generate database structure

Impact	Category	Report	Approved Change	Reference
	Exiting	FS009	Spec Ed Exit: Modified procedure so that when states update the collection period in Toggle, the ETL uses the collection period dates. Spec Ed Exit: Updated join to capture students at LEA-level only to get EL status.	CIID-4920 CIID-5019
	Discipline	FS005 FS006 FS007 FS009 FS088 FS143 FS144	Fixed issue with Reports Library so Disciplinary Removals of Children with Disabilities (IDEA) Ages 3-21 now works correctly. Corrected Staging-to-Fact Discipline Issues and improved migration performance.	CIID-4838 CIID-5053
	Assessment	FS175 FS178 FS179 FS185 FS188 FS189	Corrected issue with Assessment Files (175, 178, 179, 185, 188, 189) to allow data to display for 2022 Corrected issue with Assessment Files (175, 178, 179, 185, 188, 189) in the Create Report data	CIID-4875 CIID-4921
	Homeless	FS118	Homeless Students Enrolled The CIID team has been developing “end-to-end” tests for each EdFacts report to validate the migration processes. An end-to-end test for this report was created, executed and verified.	CIID-4900
	Directory	FS029	Corrected an issue when a NCES ID for a School is the same as another School's State Identifier Directory. Generate will correctly count the student for the School for which they are enrolled. IDEA Develop Staging-to-RDS ETL Script for Directory Counts	CIID-4827 CIID-4516
	Students Involved with Firearms	FS086	Created and completed end-to-end ETL Testing for Students Involved with Firearms	CIID-4827
	User Interface		Updated data visuals in reports library	CIID-4020
	General		Complete the ESS Metadata Auditing Generate uses “metadata” to know what columns are to appear on each EdFacts report. As changes are made to EdFacts file specifications, changes may be needed in Generate to produce the reports according to the specifications	CIID-4924
	General		Fixed an issue where Get_CountSQL Clause was producing Slow Results on School Level Submission File	CIID-4862
	General		Fixed an issue where Migrate_DimLeas set the RecordEndDateTime to one day earlier than it should	CIID-4911
	General		Remove Ed-Fi Procedures from Staging schema in database Remove Staging.Migrate_Data_Validation procedure. For Ed-Fi procedures for Generate, please see Ed-Fi Repository NOTE: If your Source to Staging ETLs refer to this procedure you will need to remove the reference to it.	CIID-4916
	General		Updated Staging.Staging-to-DimSeas so it now populates DimK12Staff for State Officer	CIID-5024
	General		Updated RDS Wrapper Scripts to use new Staging-to-DimSeas, Staging-to-DimLeas, Staging-to-DimK12Schools	CIID-5025
	General		EUT (Education Unit of Total) changes required in Metadata	CIID-5054
	Exiting	Standard Report-Exit from Special Education	Modified the code for this report on exits from Special Education. **the Migrant category in this report needs additional corrections that will be available in a future release.	CIID-4837

GENERAL SECTION

Category	Software	Approved Change	Reference	Type of Impact
Upgrade	ASP.NET	Upgrade Generate from 3.1 .Net Core to 6.0 .Net Core	CIID-4898	Performance

UPGRADE PROCEDURE

No changes have been made to the Generate upgrade procedure for this release. Follow the standard Generate upgrade process to install version 5.1.

COMPATIBLE SYSTEMS

Generate was tested on the following operating systems and browsers:

• Windows 10 Pro

• Google Chrome, Version 68 0.3440.106 (Official Build) (64-bit) and Firefox Quantum 61.01 (64 bit)