Master the fundamental concepts of data management, validation, verification, and database operations
Understanding the levels of data organization from fields to databases
GIGO principle and the importance of data quality control
8 types of validation checks and error detection methods
Methods to ensure data accuracy through double-checking
Organisation, storage, and retrieval of data in databases
Comparing sequential and direct access methods
The smallest unit of data. A single piece of information about an entity. Example: "Name", "Age", "Email"
A collection of related fields. All information about a single entity. Example: One student's complete information
A collection of related records. All entities of the same type. Example: All students in a school
A collection of related files/tables. A complete information system. Example: School management system
| Feature | Database | Spreadsheet |
|---|---|---|
| Data Structure | Multiple related tables | Single worksheet or workbook |
| Data Volume | Large volumes (millions of records) | Limited (thousands of rows) |
| Relationships | Complex relationships between tables | Simple or no relationships |
| Multi-user Access | Concurrent access supported | Limited concurrent access |
| Data Integrity | Strong validation and constraints | Basic validation only |
Software that manages databases. Examples: MySQL, Oracle, Microsoft SQL Server. Provides tools for creating, maintaining, and querying databases.
Visual interface for interacting with databases. Makes database operations accessible without knowing SQL. Examples: forms, buttons, menus.
When unreliable or incoherent data is input to a data analysis, the output information will not be useful. This principle emphasizes the importance of data quality.
Example: If a student's age is entered as "200" instead of "20", any analysis based on this data will be meaningless.
Errors that exist in the original source data. Example: A survey respondent provides incorrect information.
Errors made when copying data from one medium to another. Example: Typing "1000" instead of "100" when entering data from a form.
Errors where two digits or characters are swapped. Example: Typing "12345678" instead of "12346578".
Automatic checking by computer to ensure data is reasonable, sensible, and within acceptable boundaries.
Checking to ensure that data has been accurately copied from the original source to the computer system.
Ensures that essential data is entered. Example: Email field must not be empty.
Checks if data matches a predefined set of values. Example: Gender must be "M" or "F".
Ensures data is of the correct type. Example: Age must be a number, not text.
Checks if data falls within acceptable limits. Example: Age must be between 0 and 120.
Ensures data has correct length. Example: HKID must be exactly 8 characters.
Checks if data matches a specific pattern. Example: Email must contain "@" and "."
Ensures data is logically consistent. Example: End date must be after start date.
Ensures data is unique in the system. Example: Student ID must not already exist.
Hong Kong Identity Card uses check digit for validation. Enter the first 7 characters of an HKID:
A method to detect errors in data transmission. An extra bit is added to ensure the total number of 1s is even (even parity) or odd (odd parity).
Example:
Data: 1011001 (four 1s - even)
Even parity bit: 0 (to keep total even)
Transmitted: 10110010
Provides fixed value choices. Prevents type and format errors. Example: Country selection.
Single choice from options. Good for fixed value check. Example: Gender selection.
Multiple choices allowed. Useful for presence check. Example: Terms and conditions.
Ensures correct date format. Prevents format and type errors. Example: Birth date selection.
Whole numbers without decimal points. Example: 42, -17, 0
Numbers with decimal points. Example: 3.14, -0.5, 2.0
Text data. Can contain letters, numbers, symbols. Example: "Hello", "A123"
True or False values. Used for logical conditions. Example: True, False
User enters the same data twice (e.g., password confirmation). System compares both entries. If they match, data is likely correct.
Two different people enter the same data independently. System compares both versions. Discrepancies are flagged for review.
Visual check by comparing entered data with original source. Can be done on screen or printed copy. Human verification method.
| Aspect | Validation | Verification |
|---|---|---|
| Purpose | Check if data is reasonable | Check if data is accurate |
| Method | Automatic computer checks | Manual comparison or double entry |
| When Applied | During data entry | After data entry |
| Example | Range check: Age 0-120 | Re-enter email address |
| Limitation | Cannot detect all errors | Time-consuming and costly |
Structuring data in tables with defined fields and data types.
Example:
Table: Students
Fields: StudentID (Integer), Name (String), Age (Integer), Email (String)
Saving data permanently in the database.
SQL Example:
INSERT INTO Students (StudentID, Name, Age, Email)
VALUES (1001, 'John Chan', 16, '[email protected]');
GUI Method: Fill in a form and click "Save" button
Searching and extracting data from the database.
SQL Example:
SELECT Name, Age FROM Students
WHERE Age >= 16
ORDER BY Name;
GUI Methods:
• Sorting: Click column header to sort
• Filtering: Enter search criteria in filter box
An index is a data structure that improves the speed of data retrieval operations. It works like a book index - instead of searching through every page, you can jump directly to the relevant section.
How it works:
1. Index stores field values and their locations
2. When searching, database checks index first
3. Index points directly to the data location
4. Much faster than scanning entire table
Example: Finding a student by StudentID with index takes milliseconds instead of seconds.
Data is accessed in order from beginning to end, one record after another.
Storage Medium: Magnetic tape
Advantages:
• Simple and inexpensive
• Good for backup and archiving
• Efficient for processing all records
Disadvantages:
• Slow for finding specific records
• Must read through all previous data
• Not suitable for random access
Data can be accessed directly without reading through previous records.
Storage Media: Hard disk, USB flash drive, Optical disk (CD/DVD)
Advantages:
• Very fast access to any record
• Suitable for interactive applications
• Can update individual records easily
Disadvantages:
• More expensive than sequential
• More complex technology
• Requires indexing system
| Attribute | Sequential Access | Direct Access |
|---|---|---|
| Access Speed | Slow for specific records | Fast for any record |
| Storage Medium | Magnetic tape | Hard disk, USB, CD/DVD |
| Cost | Lower | Higher |
| Best Use | Backup, batch processing | Interactive applications |
| Update Method | Must rewrite entire file | Can update individual records |
Click on any card to flip it and see the definition