盲人導航輔助系統 影像辨識

盲人輔具開發會考慮到視覺設計，除了因為有些法定盲人並非全盲，還有就是照護者的需求，需要快速檢視導引的可靠性，另外就是工程師debug的需要，需要用來快速檢視LLM 對影像的辨識與空間理解的能力

系統能夠提供 multimodal （文字，語音，圖像）讓使用者依照自己優勢去選擇是 universal design 的精神

生成盲人步行所需要的空間導航資訊(狹窄的巷弄)

share

生成靜態障礙圖

生成路徑

動態highlight障礙點

forward-facing

aesthetic

Component reuse and remix （lots of examples)

語音導航(同步顯示目前導航路徑位置點)

動態highlight障礙點

perspective

birds' eye and forward-facing

圖庫

健康照護科技專題二： AI 時代的輔助科技設計與實作 (Syllabus 2024 Fall)

本課程使用 Claude.ai Sonnet 3.5 Artifact, ChatGPT4o， 等先進 LLM 工具實作 AI 時代的輔助科技設計。這些 AI 工具也可廣泛使用在學術，職場，生活各層面的服務設計。

開場  運算，智慧與演化 9/15

練習: 演算法

遊戲復健 9/22

Whack-a-Mole，Merry-go-round，StarTrek，Squirrel Finder

練習: 製作遊戲

UI 設計 9/29

練習: UI 設計

AI 時代下研究方法的典範轉移 10/6

LLM 與深度學習

AI視覺輔助科技 

練習: 空間智慧

空間資訊與識別輔助 10/13

floor plan for blind navigation

空間素描與空間地圖

練習: 盲人空間智慧

聲景地圖 10/20

盲人導航系統簡易UI 設計，

盲人導航影像辨識

盲人導航像辨識2 (safe route)

Autopilot

練習: 盲人AI導航

訊息處理，心智障礙生活輔助 10/27

練習：輔具設計

期中考週停課一次 11/3

ＡＩ輔助情感分析 11/10

練習：情感，情緒，人工智慧

進階Claude 生成技巧  11/17

LLM 的優勢與劣勢

練習：進階AI運用

ＡＩ輔助論文撰寫  11/24

ＡＩ輔助情感分析(II)  12/1　（界線主題app )

ＡＩ輔助媒體創作

• AI+VR+Gaming 科技大趨勢的服務應用

練習：G4H app 製作，媒體創作

生成式AI在環境安全的應用 12/8

渡河問題

Safety Visualization 

Safety Visualization for All People

AI視覺與大型語言模型在無障礙空間檢測與安全路徑規劃的應用 12/15

無障礙空間檢測

AI Vision and LLM for Safe Routing Recommendation 

AI Vision and LLM for Safe Routing Planning OmniSafe (Voronoi-based)

end of courses

• Obstacle Avoidance 2
• Obstacle Avoidance 1
• OmniSafe: Follow Voronoi 4 (Rural Road)
• OmniSafe: Follow Voronoi 3 (Country Alley)
• OmniSafe: Follow Voronoi 2 (New Scenario Corridor)
• OmniSafe: Follow Voronoi 1 (New Scenario Campus Trail)

Operational Research

• 狗兒池邊取球不下水
• River crossing interactive visualization

• Reasoning vs. Heuristics for AI
• A* search
• River crossing interactive visualization 2
• Strange family crossing a river 3

期末分享 12/22

期末分享 12/29

期末考周停課一次 1/5

 人才培育: 未來我們需要的人才，學校教育目標

生命素養

• 懂得生活，享受生命
• 同理心，尊重生命
• 完整的情緒，完整的人
• 面對低谷，心理素質

人際關係

• 可被信賴，信賴別人
• 可以帶來溫暖給自己和他人
• 幫助別人也接受別人幫助

身心健康

• 運動能力
• 懂得接地，回春

藝術與美學

• 藝術鑑賞能力
• 美學素養

自我實現

• 自我認知
• 內在召喚
• 生涯發展

健康照護科技專題一： AI 時代的輔助科技設計與實作 (Syllabus 2025 Spring)

本課程使用 Claude.ai Sonnet 3.7 Artifact, ChatGPTo1， 等先進 LLM 工具實作 AI 時代的輔助科技設計。這些 AI 工具也可廣泛使用在學術，職場，生活各層面的服務設計。

2/18/2025

2/25/2025

開場 3/4

DeepSearch and Report 3/11

出國行程規劃

中子星雙星對撞(AI科學模擬)

HW#4

• 
  

3/25

極紫外光（EUV）光源, IEEE Spectrum The Tiny Star Explosions Powering Moore’s Law 4/1

科學動畫 The Schrödinger equation

光合作用

嗅覺原理

HW#6

4/8 科學模擬

AI大數據局勢分析

HW#7

期中考週停課一次 4/15

 4/22 AI 論文寫作與研究創新

HW#8

4/29

Revise papers

物理模擬

HW#9

5/6

Financial simulation 

HW#10

5/13

Recover systems for documents

HW#11 

5/20

Game Design using AI

HW#12

5/27

Complex simulations astronomy

Cat coat colors

HW#13

6/3

Fractals

6/10

Vibe Coding

HW#15 UI 設計

期末考周停課 6/17

空間資訊與識別輔助

LLM & VLM for Blind Navigation (text)

周圍環境旁白

Abandoned Campus Building Reclaimed by Nature

學院的時空故事

社區中心室內

non-blind, blind 場景描述

圖書館內部描述(non-blind, blind) alive 盲人資訊

觸感，聲音，氣味，安全，詢問，建議(有聲書)

社區圖書館內部描述(non-blind, blind) alive 盲人資訊

依空間結構描述

Prompting as a process

提示(Prompt)撰寫其實是一個逐步修正的過程，不要期待第一次就上手

1. 迭代式提示開發： 創建有效的提示通常需要多次迭代。初始提示可能無法完全符合預期的結果。因此，根據初始輸出對提示進行精煉至關重要。

2. 
   

3. 清晰和具體： 在提示的指令中清晰和具體很重要。無論是字數、句子結構還是特定所需的詳細資訊，都提供清晰的指導對於模型獲得更準確的結果至關重要。

4. 
   

5. 調整提示： 如果初始輸出不符合期望（例如，長度過長或缺少特定資訊），則需要修改指令。這可能涉及調整字數、添加或更改特定細節，或改變輸出的焦點。

6. 
   

7. 複雜的提示： 隨著時間的推移，隨著迭代的進行，提示可能變得更加詳細和複雜。這種複雜性可能包括格式要求（如HTML輸出）或要求特定的數據安排（如表格）。

8. 
   

9. 應用軟體的成熟度： 隨著應用的成熟，對提示進行評估，應對較大量的示例進行評估變得至關重要，以確保在不同的輸入下具有一致且準確的性能。

10. 
    

11. 整體而言，透過迭代方法開發提示，從單一示例開始，逐步根據輸出精煉指令，有助於為利用LLM的各種應用程式打造有效的提示。

Example: Test of comprehension 

Version 1: Generate the problems

Prepare 10 multiple choice problems from the paste text above to test comprehension.

1. Shani Dhanda's activism primarily focuses on:
   A) Raising awareness for mental health issues
   B) Advocating for environmental sustainability
   C) Promoting cultural diversity
   D) Advocating for the rights of people with disabilities
   What year did Shani Dhanda organize the first event supporting disabled individuals and South Asian women?
   A) 2018
   B) 2019
   C) 2020
   D) 2021
   What medical condition does Shani Dhanda have?
   A) Rare genetic disorder
   B) Brittle Bone Disease
   C) Chronic fatigue syndrome
   D) Muscular dystrophy
   The "Diversability Card" app created by Shani Dhanda aims to provide:
   A) Social media networking for disabled individuals
   B) Travel guidance for people with disabilities
   C) Restaurant ratings and food delivery services
   D) Discounts for people with disabilities
   Shani Dhanda's vision for the future primarily revolves around:
   A) Global entrepreneurship programs
   B) Financial independence for disabled individuals
   C) Reducing carbon emissions
   D) Increasing government funding for disability services

2. 
   

Version 2: Making it more interactive

Prepare 10 multiple choice problems from the paste text above to test comprehension. Generate one problem at a time. Then you request an answer. Then generate next problem.

Version 3 Adding the grading 

Prepare 10 multiple choice problems from the paste text above to test comprehension. Generate one problem at a time. Then you request an answer. After you receive one, grade it. Then generate next problem. 

Version 4 Calculating the total score

Prepare 10 multiple choice problems from the paste text above to test comprehension. Generate one problem at a time. Then you request an answer. After you receive one, grade it. Then generate next problem. In the end, tell me out of the 10 questions above, how many questions are correctly answered？

Example: Test of vocabulary

Version 1 Generate the problems

Based on the vocabulary: (Exoskeleton, Myoelectric, Sensors, Hydraulic, Cylinder, Prosthesis, Rehabilitation, Paralysis, Spinal cord, Neural),  prepare 5 fill the blank quiz questions. 

1. The ___________ system in robotics uses electrical signals generated by muscle contractions to control artificial limbs or devices.
2. Sensors integrated into exoskeletons enable the detection of motion and force, allowing for precise ___________ control in assisting movements.
3. A ___________ cylinder is a mechanical device that uses fluid power to perform various tasks, often employed in advanced prosthetic designs for enhanced functionality.
4. ___________ is a process that aims to restore functions to the body after an injury, commonly employed after amputation to adapt to a new prosthesis.
5. Damage to the ___________ cord can result in loss of sensation or motor function below the injury site, leading to conditions such as paralysis.

Version 2 One at a time

Based on the vocabulary: (Exoskeleton, Myoelectric, Sensors, Hydraulic, Cylinder, Prosthesis, Rehabilitation, Paralysis, Spinal cord, Neural),  prepare 5 fill the blank quiz questions.  Generate one problem at a time. Then you request an answer.  Then generate next problem. 

Version 3 Grading answer interactively

Based on the vocabulary: (Exoskeleton, Myoelectric, Sensors, Hydraulic, Cylinder, Prosthesis, Rehabilitation, Paralysis, Spinal cord, Neural),  prepare 5 fill the blank quiz questions.  Generate one problem at a time. Then you request an answer. After you receive one, grade it. Then generate next problem. 

Version 4 Total score

Based on the vocabulary: (Exoskeleton, Myoelectric, Sensors, Hydraulic, Cylinder, Prosthesis, Rehabilitation, Paralysis, Spinal cord, Neural), prepare 5 fill the blank quiz questions. Each question has only one missing word. Generate one problem at a time. Then you request an answer. After you receive one, grade it. Then generate next problem. In the end tell me how many are correctly answered.

Version 5 Randomize

Based on the vocabulary: (Exoskeleton, Myoelectric, Sensors, Hydraulic, Cylinder, Prosthesis, Rehabilitation, Paralysis, Spinal cord, Neural), randomize the order of these words and then prepare 5 fill the blank quiz questions.  Generate one problem at a time. Then you request an answer. After you receive one, grade it. Then generate next problem. In the end tell me how many are correctly answered.

Version 6 Limiting the # of blanks to one

Based on the vocabulary: (Exoskeleton, Myoelectric, Sensors, Hydraulic, Cylinder, Prosthesis, Rehabilitation, Paralysis, Spinal cord, Neural), randomize the order of these words and then prepare 5 fill the blank quiz questions. Each question has only one missing word. Generate one problem at a time. Then you request an answer. After you receive one, grade it. Then generate next problem. In the end tell me how many are correctly answered.

based on ChatGPT and the course in https://learn.deeplearning.ai/chatgpt-prompt-eng/lesson/3/iterative