Jin-Seop Lee

I am a Ph.D. student in Artificial Intelligence at Sungkyunkwan University, working in the Information & Intelligence System Lab (IISLab), advised by Prof. Jee-Hyong Lee and co-advised by Prof. JinYeong Bak. Previously, I was a visiting researcher at Nanyang Technological University, Singapore, where I worked with Prof. Boyang Albert Li and Prof. Jaehong Yoon. I received my B.S. degree from Sungkyunkwan University.

profile photo

I aim to build a J.A.R.V.I.S.-like Proactive AI assistant that integrates visual cues such as gaze and individual user personas, while autonomously identifying and continually developing proactive assistance skills. My research includes:

Proactive AI

Proactive Assistance, Intervention Timing, Personalized Guidance

Self-Evolving AI

Planner Evolution, Planner–Policy Co-evolution, Continual Learning, Learning from Feedback

Video Understanding

Video Temporal Grounding, Fine-grained Video Understanding, Streaming Video Understanding, Hierarchical Procedural Understanding

Self-supervised Learning

Representation Learning, Contrastive Learning

Data-efficient Learning

Semi-supervised Learning, Weakly Supervised Learning, Domain Generalization, Learning with Noisy Labels

News

[2026.10] (New!) Preprint: Improving Proactive AI Assistance with Hierarchical Procedural Understanding.
[2026.10] (New!) Visiting Researcher at New York University (NYU) (Oct. 2026–Dec. 2026) (Mentor: Prof. Mengye Ren )
[2026.08] One paper accepted to EMNLP 2026 Findings.
[2026.06] Our CVPR 2026 paper received the Compute Gold Star (0.4%), a distinguished recognition for transparent compute reporting and outstanding research contribution.
[2026.04] One paper accepted to ACL 2026 Findings.
[2026.02] One paper accepted to CVPR 2026.
[2025.12] Visiting Researcher at Nanyang Technological University (NTU) (Dec. 2025–Feb. 2026) (Mentors: Prof. Boyang Albert Li and Prof. Jaehong Yoon )
[2025.11] One paper got accepted to AAAI 2026.
[2025.08] I received a Ph.D fellowship from NRF (National Research Foundation) of Korea.
[2025.07] Two papers got accepted to BMVC 2025.
[2025.05] One paper got accepted to ACL 2025.
[2024.12] One paper got accepted to AAAI 2025.
[2024.10] One paper got accepted to WACV 2025.
[2024.07] Two papers got accepted to ECCV 2024.
[2024.02] One paper got accepted to CVPR 2024.
[2024.01] One paper got accepted to EAAI 2024.

Preprints

(*: equal contribution, ‡: corresponding author)

First page of the ProactiveCoach preprint [P2] Improving Proactive AI Assistance with Hierarchical Procedural Understanding
Jin-Seop Lee, TaeYeon Won, SeongJun Jung, JungHoon Kim, Boyang Albert Li, Jin-Young Park, Jaehong Yoon‡, Jee-Hyong Lee‡
Preprint
[P1] CountCluster: Training-Free Object Quantity Guidance with Cross-Attention Map Clustering for Text-to-Image Generation
Joohyeon Lee*, Jin-Seop Lee*, Jee-Hyong Lee‡
Preprint

arXiv 2508.10710

Publications

(*: equal contribution, ‡: corresponding author)

2026

[C12] InterventionBench: Can LLM Agents Adapt to Mid-task User Interventions?
Jaehoon Lee, Suyoung Bae, Jin-Seop Lee, YunSeok Choi‡, Jee-Hyong Lee‡
EMNLP 2026 Findings
[C11] EXPO-SQL: Execution-based Clause-level Policy Optimization for Text-to-SQL
Jaehoon Lee, CheolWon Na, Suyoung Bae, Jin-Seop Lee, Jihyung Lee, YunSeok Choi‡, Jee-Hyong Lee‡
ACL 2026 Findings

arXiv 2606.23693

[C10] Learning to Refuse: Refusal-Aware Reinforcement Fine-Tuning for Hard-Irrelevant Queries in Video Temporal Grounding
Jin-Seop Lee*, SungJoon Lee*, SeongJun Jung, Boyang Li, Jee-Hyong Lee‡
CVPR 2026 (Compute Gold Star, 0.4%)

arXiv 2511.23151

[C9] BD-Net: Has depth-wise convolution ever been applied in Binary Neural Networks?
DoYoung Kim, Jin-Seop Lee, Noo-ri Kim, SungJoon Lee, Jee-Hyong Lee‡
AAAI 2026

arXiv 2511.17633

2025

[C8] TAG: A Simple Yet Effective Temporal-Aware Approach for Zero-Shot Video Temporal Grounding
Jin-Seop Lee*, Sungjoon Lee*, Jaehan Ahn, Yunseok Choi, Jee-Hyong Lee‡
BMVC 2025

arXiv 2508.07925

[C7] Stabilizing Open-Set Test-Time Adaptation via Primary-Auxiliary Filtering and Knowledge-Integrated Prediction
Byung-Joon Lee, Jin-Seop Lee, Jee-Hyong Lee‡
BMVC 2025

arXiv 2508.18751

[C6] DCG-SQL: Enhancing In-Context Learning for Text-to-SQL with Deep Contextual Schema Link Graph
Jihyung Lee*, Jin-Seop Lee*, Jaehoon Lee, YunSeok Choi‡, Jee-Hyong Lee‡
ACL 2025

arXiv 2505.19956

[C5] DomCLP: Domain-wise Contrastive Learning with Prototype Mixup for Unsupervised Domain Generalization
Jin-Seop Lee, Noo-ri Kim, Jee-Hyong Lee‡
AAAI 2025

arXiv 2412.09074

[C4] Feature-level and Spatial-level Activation Expansion for Weakly-Supervised Semantic Segmentation
Junsu Choi*, Jin-Seop Lee*, Noo-ri Kim, SuHyun Yoon, Jee-Hyong Lee‡
WACV 2025

2024

[C3] IGNORE: Information Gap-based False Negative Loss Rejection for Single Positive Multi-Label Learning
Gyeong Ryeol Song, Noo-ri Kim, Jin-Seop Lee, Jee-Hyong Lee‡
ECCV 2024
[C2] ExMatch: Semi-Supervised Learning with Scarce Labeled Samples with Additional Exploitation of Unlabeled Samples
Noo-ri Kim, Jin-Seop Lee, Jee-Hyong Lee‡
ECCV 2024
[C1] Learning with Structural Labels in Learning with Noisy Labels
Noo-ri Kim*, Jin-Seop Lee*, Jee-Hyong Lee‡
CVPR 2024
[J2] Automation of Trimming Die Design Inspection by Zigzag Process Between AI and CAD Domains
Jin-Seop Lee*, Tae-Hyun Kim*, Sang-Hwan Jeon, Sung-Hyun Park, Sang-Hi Kim, Eun-Ho Lee‡, Jee-Hyong Lee‡
EAAI(Engineering Applications of Artificial Intelligence) 2024 - TOP 3%, IF 8.0

arXiv 2305.16866

2023

[J1] Automatic defect classification using semi-supervised learning with defect localization
Yusung Kim, Jin-Seop Lee, Jee-Hyong Lee
IEEE Transactions on Semiconductor Manufacturing 2023

Research Experience

Visiting Researcher · New York University (NYU)

Mentor: Prof. Mengye Ren

Oct. 2026 – Dec. 2026
Visiting Researcher · Nanyang Technological University (NTU)

Mentors: Prof. Boyang Albert Li and Prof. Jaehong Yoon

Dec. 2025 – Feb. 2026

Education

Sungkyunkwan University (SKKU), South Korea

Integrated M.S. and Ph.D., Artificial Intelligence

Mar. 2021 - present
Sungkyunkwan University (SKKU), South Korea

B.S., Mechanical Engineering / Computer Engineering (Double Major)

Mar. 2015 - Feb. 2021
Incheon Science High School, South Korea
Mar. 2013 - Feb. 2015

Honors

Best Paper Award, 2026 Summer Annual Conference of the Korean Artificial Intelligence Association (KAIA)

Paper: PivotBench: Benchmarking LLM Agent Adaptation to Mid-task User Interventions

Aug. 2026
Compute Gold Star (0.4%), CVPR 2026

Distinguished recognition for transparent compute reporting and outstanding research contribution

Jun. 2026
Winner, NRF Ph.D. Fellowship, National Research Foundation of Korea

Title: Development Core Technology for Efficient Reasoning of Multimodal Generative AI Models

Sep. 2025 – Aug. 2027

Reviewer

Conferences: CVPR (2024–), ICCV (2025–), ECCV (2024–), WACV (2024–), BMVC (2025–), AAAI (2025–), ARR (2025–), NeurIPS (2026–), ICML (2026–), ICLR (2026–), ACM SAC (2024–)

Journals: International Journal of Computer Vision (2026–)


Design and source code from Jon Barron's website.