자동화 툴 - 공부해보기
https://www.youtube.com/watch?v=ywH7JIK34Tg
- yeji KimY
공부 내용
랜딩 페이지 만들기
https://www.youtube.com/watch?v=jnRqq2JEdxI https://www.youtube.com/watch?v=_6k1k7NtZRI Framer ai 프롬프트 사용법
- yeji KimY
패턴러닝
패턴 러닝 !! template matching ... 궁금한 것들 패턴 매칭 ... 아 이거 진짜 간단한 건데. 어 ... 이거 LLM 못 시키나. 결국 LLM을 만들어야 ... 아니 하드웨어를 만들어야 ... 똑똑한 친구를 어떻게 현업에 적용할지 ...! ...!!...!! OCR한 결과, 문법 교정한 결과 제공 사용자에게 표시해달라 LLM한테 나눠달라 고쳐달라 오 이 피드백 과정을 좀 자동화할 수 있나 좋은데 오 개좋은데? 이거를... 이거를 헤헷 요렇게 피드백하는 것만이라도 왓다갓다할 . 수있게 하고 싶다!! 헤헤헿 이걸 하려면 어떤 함수들이 필요한지 !!!
- yeji KimY
Large Language Models for Information Retrieval: A Survey
Introduction retrieval upstream - query reformulation downstream - reranking and reading reranking only on a limited set of relevant documents personalization, diversification Background Info retrieval relevance estimation - lexical similarity between the query and document vectors. Components Query rewriter Retriever Reranker - fine-grained reordering Reader - comprehend real-time user intent and generate dynamic responses Search agent LLMs Query rewriter Rewriting scenario
- yeji KimY
Web 3.0
web 1, 2, 3 1.0 - 읽기 2.0 - 읽기+쓰기 3.0 - 읽기+쓰기+개인화(소유) 탈중앙화. 플랫폼 없어짐. 시맨틱 웹 컴퓨터가 사람 대신 정보를 읽고, 이해하고, 가공하여 새로운 정보 생성. 마크업 - XML, RDF 등 온톨로지
- yeji KimY
ELK stack (Elastic search, log stash, kibana)
Elastic search - 데이터 저장, 검색 엔진 inverted index (key - word, value - doc) 특징 scale out - 샤드를 통해 수평적으로 규모를 늘릴 . 수있음 고가용성 - replica를 통해 데이터의 안정성을 보장 schema free - json 문서를 통해 데이터 검색을 수행. → 스키마 개념이 없음. rest ful - 데이터 crud 작업은 http restful api를 통해 수행하며 각각 다음과 같이 대응. ES의 검색 쿼리 컨텍스트 연관성을 계산해 최대한 비슷한 데이터를 찾아줌. BM25 필터 컨텍스트 Log stash - 데이터 수집 로그 - 반정형 데이터 로그 수집 후 로그 형태를 분석하고 정제하는 작업이 필요함. 특징 플러그인 기반 모든 형태의 데이터 처리 성능 - 자체 내장 메모리와 파일 기반 큐 사용 안정성 - 데드 레터 큐
- yeji KimY
The organization of information
Organization of recorded information The nature of information 유용성 - data<information<knowledge<understanding<wisdom Organization of information in different contexts Libraries Descriptive cataloging creating a description choosing access point ensuring authority control Subjective cataloging conceptual analysis - aboutness. translation - aboutness → controlled subject language choosing controlled vocabulary terms choosing classification notations Retrieval tools The basic retrieval tools, their formats, and their functions Bibliographies list of resources Catalogs individual items within collections of information resources Indexes
- yeji KimY
인지 부하 디자인
정보 시각화와 지식 시각화 정보 시각화 - 이해를 돕기 위해 지식 시각화 - 통찰력 전달, 새로운 지식 단위 생성 Hick의 법칙 선택 사항 최소화 복잡한 작업 단계 나누기 권장 옵션 강조 점진적인 보드 추상화 지점을 단순화 X 적절한 선택을 적시에 제공. https://tammist.tistory.com/46
- yeji KimY
.
모델 용량 - 10억으로 나눈 뒤 데이터 타입 바이트 수 곱하기 e.g. 7B 모델 16비트(2바이트) → 7*2=14GB from llama_index.core import Document, VectorStoreIndex 상호 순위 조합 구현하기 BM25와 의미 벡터 검색 방법 벡터 데이터베이스 벡터 라이브러리 - Faiss, Annoy, NMSLIB, ScaNN 벡터 전용 데이터베이스 - pinecone, weaviate, milvus, chroma, qdrant, vespa 벡터 기능 추가 데이터베이스 - elasticSearch, PostgreSQL, MongoDB, Neo4j MLOps 데이터 준비 → 모델 학습 → (모델 저장소 ↔ 모델 평가) → 모델 배포 → 모니터링 → 재학습 ... 멀티 에이전트 - AutoGen, MetaGPT, CrewAI 사용자 맞춤형 정보
- yeji KimY
DB table 구성하기
To efficiently implement the features you described, the database structure needs to account for several types of entities: lecture notes, textbooks, past exam questions, professors, and study tips. Additionally, the system should allow cross-referencing between these entities, version control for lecture notes, and the ability to filter data by professor. Here’s how you can structure the database tables: Key Tables for the Database Structure Subjects (subjects): Stores information about each subject. Professors (professors): Stores information about the professors. Lecture Notes (lecture_notes): Stores metadata for each lecture note document. Lecture Slides (lecture_slides): Stores content for individual slides within a lecture note. Textbooks (textbooks): Stores metadata about textbooks for each subject. Textbook Pages (textbook_pages): Stores the text content of each page in a textbook. Test Questions (test_questions): Stores individual test questions. Slide to Textbook Mapping (slide_textbook_mapping): Relates specific slides to textbook pages or paragraphs. Slide to Test Question Mapping (slide_question_mapping): Relates specific slides to test questions. Test Question to Textbook Mapping (question_textbook_mapping): Relates specific test questions to textbook pages or paragraphs. Study Tips (study_tips): Stores study tips from professors related to each subject. Versions (lecture_note_versions): Manages versioning for the lecture notes. Table Structures and Relationships Subjects Table (subjects) subject_id: INT (Primary Key) subject_name: VARCHAR(255) (e.g., "Biochemistry") description: TEXT (optional description) Professors Table (professors)
- yeji KimY
OpenAI Embedding & Semantic search
https://platform.openai.com/docs/api-reference/embeddings
- yeji KimY
OpenAI API
https://platform.openai.com/docs/guides/batch/getting-started Batch API 이용하기 Upload batch file Creating the batch Checking the status of a batch Retrieving the results 다른 LLM 알아보기 이걸로 기본적인 처리는 해야겠다!!! (와)
- yeji KimY
GPT fine tuning
https://platform.openai.com/docs/guides/fine-tuning/preparing-your-dataset https://www.datacamp.com/tutorial/fine-tuning-openais-gpt-4-step-by-step-guide finetuning GPT 4o jsonl format의 데이터를 만들어야 함. {"prompt": "<prompt text>", "completion": "<ideal generated text>"} {"messages": [{"role": "system", "content": "You are teaching assistant for Machine Learning. You should help to user to answer on his question."}, {"role": "user", "content": "What is machine learning?"}, {"role": "assistant", "content": "'Tis but the art of teaching machines to think, to learn from data most fine, and decisions to link."}]} {"messages": [{"role": "system", "content": "You are teaching assistant for Machine Learning. You should help to user to answer on his question."}, {"role": "user", "content": "How doth neural networks work?"}, {"role": "assistant", "content": "They're like the brains of yon digital beast, with layers many, and nodes that cease."}]} Upload training file Create a fine-tuned model Analyze fine-tuned model UI로도 관련 정보를 볼 수 있음. fine tuning 작업이 끝나면, job details의 'fine_tuned_model' 필드에서 모델 이름을 볼 수 있음. → 아래아 같이 모델 이름을 적고 활용하면 됨.
- yeji KimY
Semantic search
ENHANCING KNOWLEDGE RETRIEVAL WITH IN-CONTEXT LEARNING AND SEMANTIC SEARCH THROUGH GENERATIVE AI Method 1 : Generative text retrieval (GTR) 각 chunk에 word2vec 등으로 Embedding → vector database 구축 query embedding과 유사도 계산 → 가장 가까운 걸로. Generative tabular text retrieval (GTR-T) 먼저 database table과 meta data를 가져와서 .csv로 저장. query를 embedding하여 관련있는 table을 찾음 이 table을 Llm한테 줘서 적절한 sql 쿼리문을 생성함. Olio: A Semantic Search Interface for Data Repositories Intro Q&A, exploratory search, design search. 태블로를 활용한 시각화 → 썸네일 제공? Related works Semantic web search system keyword(structured query languages) based or NL based keyword based QUERIX - stanford CoreNLP parser + wordNet olio는 trends, location, groupings, aggregations, filters 등으로 intentfmf qnsfbgka. 의도를 특정 그룹으로 나누는 것 같음. 내가 하려는 것과 잘 어울리는지는 모르겠어서 일단 읽기 중단. Know where to go : make llm a relevant, responsible, and trustworthy searcher. Intro
- yeji KimY
하이퍼레저 패브릭 - 체인코드. - 스마트 컨트랙트 역할. 여러가지 플러그 기ㅏ능 옵션. 원장 데이터는 다양한 형식으로 저장. 컨센서스도. - kafka, raft. msp 목표 허가된 참여자 대상 모듈러 아키텍쳐 기반. 특징 퍼미션드 비결정적. 교체 가능한 모듈러 아키텍처 가능한 컨센서스 알고리즘 - solo, kafka, practical byzanti
하이퍼레저 패브릭 - 체인코드. - 스마트 컨트랙트 역할. 여러가지 플러그 기ㅏ능 옵션. 원장 데이터는 다양한 형식으로 저장. 컨센서스도. - kafka, raft. msp 목표 허가된 참여자 대상 모듈러 아키텍쳐 기반. 특징 퍼미션드 비결정적. 교체 가능한 모듈러 아키텍처 가능한 컨센서스 알고리즘 - solo, kafka, practical byzantine fault tolerant 멀티 블록체인 하나의 블록체인 네트워크를 논리적으로 독립된 여러 개의 블록체인으로 분할 가능. 순서화가 왜 필요하지? 컨센서스 알고리즘? 멀티 블록체인 지원? 컨소시엄 블록체인 - 허가된 기관만. 컨소시엄 소속 참여자가 관리 주체. r3cev, casper 프라이빗 - 허가된 기관만. 중앙 기관이 모든 권한 보유. hl fabric, eea
- yeji KimY
하이퍼레저 패브릭 개발 - dapp 실행 플로우 (네트워크 구동)
https://www.youtube.com/watch?v=VAjOIZB4PVI 회사 컨소시움 구성 - a, b, c 회사 각 회사에서 peer 몇개 할지. a-peer 1, b-peer 2개 ... 회사마다 ca_a, ca_b, ca_c → root_ca cdb(couch db) orderer 기관 결정. (solo, kafka - 한개 이상, etcdraft - 비동기?) 채널이라는 소그룹 - 채널 트랜잭션 처리 hl fabric을 이용하면 여러 요소들을 가져다가 쉽게 네트워크를 구성할 수 있음. mongo DB -
- yeji KimY
궁금한 것 보증 정책 : 주어진 스마트 컨트렉트에 의해 생성된 거래에 서명해야 하는 블록체인 네트워크의 조직 for 해당 거래가 유효하다고 선언 → 유효하다는 선언은 어떻게 하지? 유효 여부에 관계없이 모든 거래는 블록체인에 기록되지만 유효한 거래만 world state에 기여 ex 자동차 이체 거래 t3 : ORG1과 ORG2 사이의 자동차 환승에 대한
궁금한 것 보증 정책 : 주어진 스마트 컨트렉트에 의해 생성된 거래에 서명해야 하는 블록체인 네트워크의 조직 for 해당 거래가 유효하다고 선언 → 유효하다는 선언은 어떻게 하지? 유효 여부에 관계없이 모든 거래는 블록체인에 기록되지만 유효한 거래만 world state에 기여 ex 자동차 이체 거래 t3 : ORG1과 ORG2 사이의 자동차 환승에 대한 트랜잭션 입력 : {CAR1, ORG1, ORG2} 출력 : {CAR1.owner=ORG1, CAR1.owner=ORG2} => ORG1에서 ORG2로 소유자가 변경되었음을 나타내는 방법 애플리케이션의 조직 ORG1에서 입력이 서명 보증 정책 ORG1 및 ORG2로 식별된 두 조직에서 출력이 서명 서명은 개인 키를 사용하여 생성 네트워크의 모든 노드가 트랜잭션에 대해 동의, 네트워크의 모든 사람이 확인 가능 트랜잭션 : 두 단계의 검증 보증 정책에 따라 충분한 조직에서 서명했는지 확인 world state의 현재 값이 보증 피어 노드에 의해 서명되었을 때 트랜잭션의 읽기 세트와 일치하는지 확인 즉, 중간에 업데이트가 없었는지 확인 트랜잭션이 이 테스트를 모두 통과하면 유효한 것으로 표시됩니다. 조직이 서로 다른 상대방과 작업 트래픽을 분리하는 데 도움이 될 만큼 충분히 독립적 필요할 때 독립적인 활동을 조정할 수 있도록 충분히 통합됨. 대부분의 시나리오에서 위에서 설명한 체인코드 수명 주기 메커니즘을 사용하는 대신 체인코드에 초기화 논리를 포함 권고
- yeji KimY
DB 변경 생길 때마다 알림 주기 - MySQL trigger, flutter cron
GPT To receive notifications when a change occurs in the user's schedule value in the database, you can implement several approaches depending on your specific needs and the technologies you're comfortable with. Here’s a high-level overview of the options: 1. Polling (Active Checking) How it works: You can create a script that periodically checks the database for changes. If a change is detected, the script sends a notification. Steps: Set up a CRON job (or Windows Task Scheduler if on Windows) that runs a script every X minutes. The script queries the database for changes in the user's schedule value. If a change is detected, send a notification (e.g., via email, SMS, or push notification). Pros: Easy to implement. Cons: Not real-time, may introduce unnecessary load on the server if the polling frequency is too high. 2. Database Triggers + Notification System How it works: Use a database trigger to detect changes and invoke a notification system. Steps: Create a Trigger: Set up a trigger in your MySQL database (if you're using MySQL with XAMPP) that activates when a change occurs in the user's schedule table. Trigger Action: The trigger could insert a record into a separate "notifications" table or call an external script to send a notification. Notification System: Use a script or service (such as an email or SMS API) to send notifications when new records are added to the "notifications" table. Pros: Real-time notifications, less overhead than polling. Cons: More complex setup, requires database-level modifications. Example of a MySQL trigger: 3. WebSocket or Server-Sent Events (SSE)
- yeji KimY
피그마
frame 해상도 - 점유도 커뮤니티 ui KIT - duplicate 버튼생성하기 1-3, 프로토타이핑, text styling, auto layouthttps://www.youtube.com/watch?v=E4NfxpV9hpE 미드저니
- yeji KimY
fabric application
about asset transfer 구성 요소 샘플 app smart contract 샘플 app 준비하기 npm install → 종속성 설치, 앱 빌드 1. gateway에 대한 gRPC 연결 설정 2. gateway 연결 생성 요구사항 fabric gateway에 대한 gRPC 연결 네트워크와 거래할 때 사용되는 client ID 디지털 서명 3. 호출할 계약에 액세스 gateway.getNetwork, network.getContract 4. 샘플 자산으로 원장 채우기 submitTranscation은 fabric gateway를 통해 다음을 수행 거래 제안 승인 승인된 거래를 주문 서비스에 제출 트랜젝션이 커밋되고 원장 상태가 업데이트될 때까지 대기 샘플 앱에 initLedger 호출. contract.submitTransaction
- yeji KimY
아무튼-작업일지
공부 내용
DB table 구성하기
Y
yeji Kim
To efficiently implement the features you described, the database structure needs to account for several types of entities: lecture notes, textbooks, past exam questions, professors, and study tips. Additionally, the system should allow cross-referencing between these entities, version control for lecture notes, and the ability to filter data by professor.
Here’s how you can structure the database tables:
Key Tables for the Database Structure
1.
Subjects (subjects): Stores information about each subject.
2.
Professors (professors): Stores information about the professors.
3.
Lecture Notes (lecture_notes): Stores metadata for each lecture note document.
4.
Lecture Slides (lecture_slides): Stores content for individual slides within a lecture note.
5.
Textbooks (textbooks): Stores metadata about textbooks for each subject.
6.
Textbook Pages (textbook_pages): Stores the text content of each page in a textbook.
7.
Test Questions (test_questions): Stores individual test questions.
8.
Slide to Textbook Mapping (slide_textbook_mapping): Relates specific slides to textbook pages or paragraphs.
9.
Slide to Test Question Mapping (slide_question_mapping): Relates specific slides to test questions.
10.
Test Question to Textbook Mapping (question_textbook_mapping): Relates specific test questions to textbook pages or paragraphs.
11.
Study Tips (study_tips): Stores study tips from professors related to each subject.
12.
Versions (lecture_note_versions): Manages versioning for the lecture notes.
Table Structures and Relationships
1.
Subjects Table (subjects)
•
subject_id: INT (Primary Key)
•
subject_name: VARCHAR(255) (e.g., "Biochemistry")
•
description: TEXT (optional description)
2.
Professors Table (professors)
•
professor_id: INT (Primary Key)
•
professor_name: VARCHAR(255)
•
email: VARCHAR(255) (optional)
•
department: VARCHAR(255) (optional)
3.
Lecture Notes Table (lecture_notes)
•
lecture_note_id: INT (Primary Key)
•
subject_id: INT (Foreign Key to subjects)
•
professor_id: INT (Foreign Key to professors)
•
title: VARCHAR(255) (Title of the lecture note)
•
version: VARCHAR(50) (Version of the lecture notes)
•
upload_date: DATE (When the lecture notes were uploaded)
•
file_path: VARCHAR(255) (Path to the lecture note file, if necessary)
•
notes: TEXT (Additional notes)
4.
Lecture Slides Table (lecture_slides)
•
slide_id: INT (Primary Key)
•
lecture_note_id: INT (Foreign Key to lecture_notes)
•
slide_number: INT (Order of the slide in the lecture)
•
slide_title: VARCHAR(255) (Optional: Title or caption of the slide)
•
slide_content: TEXT (Content of the slide in text format)
•
image_path: VARCHAR(255) (Optional: Path to image associated with the slide)
•
notes: TEXT (Optional: additional notes)
5.
Textbooks Table (textbooks)
•
textbook_id: INT (Primary Key)
•
subject_id: INT (Foreign Key to subjects)
•
textbook_title: VARCHAR(255)
•
author: VARCHAR(255)
•
publication_year: YEAR
•
isbn: VARCHAR(13) (ISBN number)
6.
Textbook Pages Table (textbook_pages)
•
page_id: INT (Primary Key)
•
textbook_id: INT (Foreign Key to textbooks)
•
page_number: INT (Page number)
•
content: TEXT (Text of the page)
7.
Test Questions Table (test_questions)
•
question_id: INT (Primary Key)
•
subject_id: INT (Foreign Key to subjects)
•
professor_id: INT (Foreign Key to professors)
•
exam_year: YEAR
•
exam_type: VARCHAR(50) (E.g., "Midterm", "Final", etc.)
•
question_text: TEXT (The content of the test question)
•
correct_answer: TEXT (Optional: Correct answer)
•
image_path: VARCHAR(255) (Optional path to images associated with the question)
8.
Slide to Textbook Mapping Table (slide_textbook_mapping)
•
mapping_id: INT (Primary Key)
•
slide_id: INT (Foreign Key to lecture_slides)
•
textbook_id: INT (Foreign Key to textbooks)
•
page_id: INT (Foreign Key to textbook_pages)
•
relation_type: VARCHAR(255) (E.g., "full match", "partial match", "reference")
9.
Slide to Test Question Mapping Table (slide_question_mapping)
•
mapping_id: INT (Primary Key)
•
slide_id: INT (Foreign Key to lecture_slides)
•
question_id: INT (Foreign Key to test_questions)
•
relation_type: VARCHAR(255) (E.g., "direct", "related", "concept match")
10.
Test Question to Textbook Mapping Table (question_textbook_mapping)
•
mapping_id: INT (Primary Key)
•
question_id: INT (Foreign Key to test_questions)
•
textbook_id: INT (Foreign Key to textbooks)
•
page_id: INT (Foreign Key to textbook_pages)
•
relation_type: VARCHAR(255) (E.g., "direct answer", "related concept")
11.
Study Tips Table (study_tips)
•
tip_id: INT (Primary Key)
•
professor_id: INT (Foreign Key to professors)
•
subject_id: INT (Foreign Key to subjects)
•
tip_content: TEXT (Text content of the study tip)
•
created_at: DATETIME (Time when the tip was added)
12.
Versions Table (lecture_note_versions)
•
version_id: INT (Primary Key)
•
lecture_note_id: INT (Foreign Key to lecture_notes)
•
version_number: VARCHAR(50) (E.g., "v1.0", "v2.0")
•
uploaded_at: DATETIME (When this version was uploaded)
•
file_path: VARCHAR(255) (Optional: Path to the file of this version)
•
notes: TEXT (Optional notes for this version)
Query Examples
1.
Check Textbook Pages, Paragraphs, and Test Questions Related to Each Specific Slide in a Lecture Note
-- Fetch related textbook pages for a specific slide
SELECT tp.page_number, tp.content, t.textbook_title
FROM slide_textbook_mapping stm
JOIN textbook_pages tp ON stm.page_id = tp.page_id
JOIN textbooks t ON tp.textbook_id = t.textbook_id
WHERE stm.slide_id = ?;
-- Fetch related test questions for a specific slide
SELECT tq.question_text, tq.exam_year, tq.exam_type
FROM slide_question_mapping sqm
JOIN test_questions tq ON sqm.question_id = tq.question_id
WHERE sqm.slide_id = ?;1.
Check Lecture Slides and Textbook Contents Related to Test Questions
-- Fetch related lecture slides for a test question
SELECT ls.slide_title, ls.slide_content
FROM slide_question_mapping sqm
JOIN lecture_slides ls ON sqm.slide_id = ls.slide_id
WHERE sqm.question_id = ?;
-- Fetch related textbook pages for a test question
SELECT tp.page_number, tp.content, t.textbook_title
FROM question_textbook_mapping qtm
JOIN textbook_pages tp ON qtm.page_id = tp.page_id
JOIN textbooks t ON tp.textbook_id = t.textbook_id
WHERE qtm.question_id = ?;1.
Collect Lecture Notes and Test Questions from Each Professor
-- Fetch all lecture notes for a professor
SELECT ln.title, ln.version, ln.upload_date
FROM lecture_notes ln
WHERE ln.professor_id = ?;
-- Fetch all test questions from a professor
SELECT tq.question_text, tq.exam_year, tq.exam_type
FROM test_questions tq
WHERE tq.professor_id = ?;1.
Reflect Updates to the Lecture Notes Version
When uploading a new version of a lecture note, insert a new record into the lecture_note_versions table and update the lecture_notes
table with the latest version information.
-- Insert a new version
INSERT INTO lecture_note_versions (lecture_note_id, version_number, uploaded_at, file_path, notes)
VALUES (?, ?, NOW(), ?, ?);
-- Update the lecture note to reflect the latest version
UPDATE lecture_notes
SET version = ?, upload_date = NOW()
WHERE lecture_note_id = ?;Additional Considerations
1.
Indexes:
•
Add indexes on the foreign key columns (professor_id, slide_id, question_id, textbook_id) to optimize queries for faster retrieval.
2.
Full-Text Search:
•
Implement full-text search on fields like slide_content, question_text, and textbook_content to allow for efficient searching of large bodies of text.
3.
Versioning:
•
Each lecture note can have multiple versions. This is handled by the lecture_note_versions table. When fetching a lecture note, you can specify which version to retrieve, or you can retrieve the latest version.
4.
Language and Multilingual Support:
•
If the system needs to support multiple languages, consider adding a language_code field to relevant tables like lecture_slides, textbooks, test_questions, and study_tips to differentiate between content in different languages.
5.
Diagrams and Images:
•
Ensure that the tables include fields (image_path) for storing paths to images, diagrams, or charts that accompany the lecture notes or test questions. These visual aids are important for certain subjects like biochemistry or physics.
Summary
This database schema will efficiently support the cross-referencing of lecture notes, textbooks, and test questions. It will also allow the system to collect data by professor, track multiple versions of lecture notes, and store study tips from professors. The system is flexible enough to handle updates and provide a comprehensive resource for both students and professors.
Key Features to Implement:
Linking to Original PDF Content: Provide direct links to the specific parts of the original PDF for lecture notes, textbook pages, and test questions.
Handling Images and Photos in Test Questions: Ensure that questions containing images can display both the text and image properly.
Incorporating Tables and Pictures in Lecture Notes and Textbooks: Enable storage and display of tables and images that appear in textbook pages or lecture slides.
Updated Database Structure
Linking to Original PDF Content: Provide direct links to the specific parts of the original PDF for lecture notes, textbook pages, and test questions.
Handling Images and Photos in Test Questions: Ensure that questions containing images can display both the text and image properly.
Incorporating Tables and Pictures in Lecture Notes and Textbooks: Enable storage and display of tables and images that appear in textbook pages or lecture slides.
Updated Database Structure
1.
Additions to Existing Tables
Changes to the textbook_pages, lecture_slides, and test_questions tables:
Changes to the textbook_pages, lecture_slides, and test_questions tables:
PDF linking: Add fields to store the exact location (e.g., page number or coordinates) of content in the original PDF.
Tables and images: Add fields to store image paths or links for tables and pictures embedded in the textbook pages, lecture notes, and test questions.
Updated Table Definitions:
Subjects Table (subjects) — Unchanged from before.
Tables and images: Add fields to store image paths or links for tables and pictures embedded in the textbook pages, lecture notes, and test questions.
Updated Table Definitions:
Subjects Table (subjects) — Unchanged from before.
Professors Table (professors) — Unchanged from before.
Lecture Notes Table (lecture_notes)
lecture_note_id: INT (Primary Key)
subject_id: INT (Foreign Key to subjects)
professor_id: INT (Foreign Key to professors)
title: VARCHAR(255) (Title of the lecture note)
version: VARCHAR(50) (Version of the lecture notes)
upload_date: DATE
file_path: VARCHAR(255) (Path to the original PDF)
notes: TEXT (Optional extra notes)
Lecture Slides Table (lecture_slides)
subject_id: INT (Foreign Key to subjects)
professor_id: INT (Foreign Key to professors)
title: VARCHAR(255) (Title of the lecture note)
version: VARCHAR(50) (Version of the lecture notes)
upload_date: DATE
file_path: VARCHAR(255) (Path to the original PDF)
notes: TEXT (Optional extra notes)
Lecture Slides Table (lecture_slides)
slide_id: INT (Primary Key)
lecture_note_id: INT (Foreign Key to lecture_notes)
slide_number: INT (Order in the lecture)
slide_title: VARCHAR(255) (Optional)
slide_content: TEXT (Content of the slide in text format)
image_path: VARCHAR(255) (Optional path to images used in the slide)
pdf_page: INT (Page number of the slide in the original PDF)
pdf_coordinates: VARCHAR(255) (Optional: Coordinates for specific regions in the PDF, if needed for deep linking)
table_data: TEXT (Optional: JSON or serialized format for any tables in the slide)
notes: TEXT (Additional notes)
Textbooks Table (textbooks) — Unchanged from before.
lecture_note_id: INT (Foreign Key to lecture_notes)
slide_number: INT (Order in the lecture)
slide_title: VARCHAR(255) (Optional)
slide_content: TEXT (Content of the slide in text format)
image_path: VARCHAR(255) (Optional path to images used in the slide)
pdf_page: INT (Page number of the slide in the original PDF)
pdf_coordinates: VARCHAR(255) (Optional: Coordinates for specific regions in the PDF, if needed for deep linking)
table_data: TEXT (Optional: JSON or serialized format for any tables in the slide)
notes: TEXT (Additional notes)
Textbooks Table (textbooks) — Unchanged from before.
Textbook Pages Table (textbook_pages)
page_id: INT (Primary Key)
textbook_id: INT (Foreign Key to textbooks)
page_number: INT (Page number of the textbook)
content: TEXT (The text of the page)
pdf_page: INT (Page number in the original textbook PDF)
pdf_coordinates: VARCHAR(255) (Optional: Coordinates for specific parts of the page in the PDF)
image_paths: TEXT (Optional: Comma-separated list of image paths extracted from the page)
table_data: TEXT (Optional: JSON or serialized format for any tables in the page)
Test Questions Table (test_questions)
question_id: INT (Primary Key)
subject_id: INT (Foreign Key to subjects)
professor_id: INT (Foreign Key to professors)
exam_year: YEAR
exam_type: VARCHAR(50) (E.g., "Midterm", "Final")
question_text: TEXT (Content of the question)
correct_answer: TEXT (Optional)
image_path: VARCHAR(255) (Optional: Path to an image included in the test question)
pdf_page: INT (Page number in the original PDF)
pdf_coordinates: VARCHAR(255) (Optional: Coordinates for the question in the PDF)
table_data: TEXT (Optional: JSON or serialized format for any tables in the question)
notes: TEXT (Optional: additional notes)
Slide to Textbook Mapping Table (slide_textbook_mapping) — Unchanged from before.
textbook_id: INT (Foreign Key to textbooks)
page_number: INT (Page number of the textbook)
content: TEXT (The text of the page)
pdf_page: INT (Page number in the original textbook PDF)
pdf_coordinates: VARCHAR(255) (Optional: Coordinates for specific parts of the page in the PDF)
image_paths: TEXT (Optional: Comma-separated list of image paths extracted from the page)
table_data: TEXT (Optional: JSON or serialized format for any tables in the page)
Test Questions Table (test_questions)
question_id: INT (Primary Key)
subject_id: INT (Foreign Key to subjects)
professor_id: INT (Foreign Key to professors)
exam_year: YEAR
exam_type: VARCHAR(50) (E.g., "Midterm", "Final")
question_text: TEXT (Content of the question)
correct_answer: TEXT (Optional)
image_path: VARCHAR(255) (Optional: Path to an image included in the test question)
pdf_page: INT (Page number in the original PDF)
pdf_coordinates: VARCHAR(255) (Optional: Coordinates for the question in the PDF)
table_data: TEXT (Optional: JSON or serialized format for any tables in the question)
notes: TEXT (Optional: additional notes)
Slide to Textbook Mapping Table (slide_textbook_mapping) — Unchanged from before.
Slide to Test Question Mapping Table (slide_question_mapping) — Unchanged from before.
Test Question to Textbook Mapping Table (question_textbook_mapping) — Unchanged from before.
Study Tips Table (study_tips) — Unchanged from before.
Lecture Note Versions Table (lecture_note_versions) — Unchanged from before.
Explanation of Changes:
PDF Linking (New Fields: pdf_page and pdf_coordinates):
PDF Linking (New Fields: pdf_page and pdf_coordinates):
These fields will allow users to directly link back to the original PDF files (lecture notes, textbook, test questions) and view the relevant sections.
pdf_page: This is the page number in the original PDF where the content is located.
pdf_coordinates: (Optional) This field can be used to store coordinates on the PDF page if the goal is to deep-link to specific sections or highlight certain areas.
Image Paths for Lecture Notes, Textbooks, and Test Questions:
pdf_page: This is the page number in the original PDF where the content is located.
pdf_coordinates: (Optional) This field can be used to store coordinates on the PDF page if the goal is to deep-link to specific sections or highlight certain areas.
Image Paths for Lecture Notes, Textbooks, and Test Questions:
image_path fields allow you to store paths to images that are part of the lecture slides, textbook pages, or test questions. This ensures that diagrams, figures, and photos are properly displayed alongside the text content.
Test questions often contain images (such as diagrams or lab results). Storing the path in the image_path field ensures that the image can be displayed when presenting the question.
Table Storage (table_data):
Test questions often contain images (such as diagrams or lab results). Storing the path in the image_path field ensures that the image can be displayed when presenting the question.
Table Storage (table_data):
table_data fields are used to store table information in a structured format. You can use JSON or another serialized format (e.g., CSV) to store tables. This ensures that tables in textbook pages, lecture notes, and test questions are retained and displayed appropriately.
Example: A table extracted from a textbook can be stored as a JSON object with rows and columns, allowing you to display it in a structured format within your app.
Query Examples for New Features
Example: A table extracted from a textbook can be stored as a JSON object with rows and columns, allowing you to display it in a structured format within your app.
Query Examples for New Features
1.
Check Relevant Parts of the Original PDF for a Lecture Slide
To retrieve the location of a lecture slide in the original PDF:
To retrieve the location of a lecture slide in the original PDF:
sql
코드 복사
SELECT ls.slide_title, ls.pdf_page, ls.pdf_coordinates
FROM lecture_slides ls
WHERE ls.slide_id = ?;
This query will return the title of the slide, the page number in the original PDF, and optionally the coordinates if provided.
코드 복사
SELECT ls.slide_title, ls.pdf_page, ls.pdf_coordinates
FROM lecture_slides ls
WHERE ls.slide_id = ?;
This query will return the title of the slide, the page number in the original PDF, and optionally the coordinates if provided.
1.
Check Relevant Parts of the Original PDF for a Test Question
To retrieve the location of a test question in the original PDF:
To retrieve the location of a test question in the original PDF:
sql
코드 복사
SELECT tq.question_text, tq.pdf_page, tq.pdf_coordinates, tq.image_path
FROM test_questions tq
WHERE tq.question_id = ?;
This query will return the question text, the page number in the original PDF, the coordinates (if provided), and the path to any associated images.
코드 복사
SELECT tq.question_text, tq.pdf_page, tq.pdf_coordinates, tq.image_path
FROM test_questions tq
WHERE tq.question_id = ?;
This query will return the question text, the page number in the original PDF, the coordinates (if provided), and the path to any associated images.
1.
Check for Tables and Images in a Textbook Page
To retrieve tables and images associated with a textbook page:
To retrieve tables and images associated with a textbook page:
sql
코드 복사
SELECT tp.page_number, tp.content, tp.table_data, tp.image_paths, tp.pdf_page
FROM textbook_pages tp
WHERE tp.page_id = ?;
This query will return the content of the page, any tables stored in the table_data field, images stored in the image_paths field, and the page number in the original PDF.
코드 복사
SELECT tp.page_number, tp.content, tp.table_data, tp.image_paths, tp.pdf_page
FROM textbook_pages tp
WHERE tp.page_id = ?;
This query will return the content of the page, any tables stored in the table_data field, images stored in the image_paths field, and the page number in the original PDF.
Handling Image and Table Display in the Application
Images: The image_path fields can store the relative path to image files (e.g., diagrams or photos in questions). When displaying content, your application can fetch and render the images in conjunction with the text content.
Images: The image_path fields can store the relative path to image files (e.g., diagrams or photos in questions). When displaying content, your application can fetch and render the images in conjunction with the text content.
Tables: Tables stored in table_data (in JSON format) can be dynamically rendered in the UI using libraries like DataTables (for HTML) or converted to a visually appealing format.
Example JSON format for table_data:
json
코드 복사
{
"headers": ["Compound", "Boiling Point", "Melting Point"],
"rows": [
["Water", "100°C", "0°C"],
["Ethanol", "78.37°C", "-114.1°C"],
["Methane", "-161.5°C", "-182.5°C"]
]
}
This format is easy to parse in the application and display as a table.
json
코드 복사
{
"headers": ["Compound", "Boiling Point", "Melting Point"],
"rows": [
["Water", "100°C", "0°C"],
["Ethanol", "78.37°C", "-114.1°C"],
["Methane", "-161.5°C", "-182.5°C"]
]
}
This format is easy to parse in the application and display as a table.
Summary of Changes
By modifying the database schema with fields for PDF linking, image paths, and table storage, you can:
By modifying the database schema with fields for PDF linking, image paths, and table storage, you can:
Allow users to access specific parts of the original PDF content (lecture slides, textbook pages, and test questions).
Handle images in test questions, lecture slides, and textbook pages, ensuring that all non-textual information is properly displayed.
Store tables in a structured format, allowing for easier display and retrieval in the application.
This approach ensures a richer experience when navigating between different types of educational content.
Handle images in test questions, lecture slides, and textbook pages, ensuring that all non-textual information is properly displayed.
Store tables in a structured format, allowing for easier display and retrieval in the application.
This approach ensures a richer experience when navigating between different types of educational content.
Subscribe to '아무튼-작업일지'
Subscribe to my site to be the first to receive notifications and emails about the latest updates, including new posts.
Join Slashpage and subscribe to '아무튼-작업일지'!
Join Slashpage and subscribe to '아무튼-작업일지'!