Learning Notebook Conversion

Objectives

• •Convert .ipynb to .py while preserving structure and documentation
• •Convert .py to .ipynb with proper cell organization
• •Maintain code quality and academic formatting during conversion

Instructions

1. Conversion Methods

Method A: VS Code Built-in (Recommended for single files)

1. •Open .ipynb file
2. •Press Ctrl+Shift+P
3. •Select "Notebook: Export to Python Script"
4. •Choose save location

Method B: Jupyter nbconvert (Recommended for batch/automation)

# .ipynb to .py
jupyter nbconvert --to python notebook.ipynb

# .py to .ipynb
jupytext --to notebook script.py

Method C: Python script (For custom processing)

import nbformat
from nbconvert import PythonExporter

# Read notebook
with open('notebook.ipynb', 'r', encoding='utf-8') as f:
    nb = nbformat.read(f, as_version=4)

# Convert to Python
exporter = PythonExporter()
source, meta = exporter.from_notebook_node(nb)

# Write to file
with open('output.py', 'w', encoding='utf-8') as f:
    f.write(source)

2. .ipynb → .py Conversion

What gets converted:

• •Code cells → Python code
• •Markdown cells → Multi-line comments ("""...""")
• •Cell separators → # %% markers (VS Code interactive cells)
• •Outputs → Removed (code only)

Post-conversion cleanup:

1. •

   Add file-level docstring (if missing):

   python

   """
   Course Code Lab X: Title
   Author: [Student Name]
   Section: [Section Number]
   Date: [Date]
   """
   

2. •

   Add configuration constants section (if needed):

   Constants should be grouped at the top of the file with the same section header format:

   python

   # ============================================================
   # 步骤0：配置常量
   # Step 0: Configuration constants
   # ============================================================
   
   # 固定随机种子以确保结果可复现
   # Fixed seed for reproducibility
   RANDOM_STATE = 42
   
   # 80-20 训练集-测试集划分
   # 80-20 train-test split
   TEST_SIZE = 0.2
   

   Key points:

   • •Use # ============================================================ as section delimiters
   • •Number the section as "步骤0" / "Step 0" to indicate it runs before main steps
   • •Each constant should have bilingual comments (Chinese on top, English below)
   • •Use UPPER_SNAKE_CASE for constant names
3. •

   Review cell markers:

   • •Keep # %% for logical sections
   • •Remove unnecessary markers
   • •Add descriptive comments after markers
4. •

   Convert markdown to docstrings:

   • •Section headers → Comments
   • •Important notes → Inline comments
   • •Instructions → Function docstrings
5. •

   Adjust outputs:

   • •Change display() to print()
   • •Ensure plt.show() for plots
   • •Remove IPython-specific magic commands

Example conversion:

# Before (in .ipynb markdown cell):
# ## Step 1: Load Data
# Load the dataset and display first 5 rows

# After (in .py):
# %% Step 1: Load Data
# Load the dataset and display first 5 rows
import pandas as pd
df = pd.read_csv('data.csv')
print(df.head())

3. .py → .ipynb Conversion

Using jupytext:

# Install jupytext
pip install jupytext

# Convert with cell markers
jupytext --to notebook script.py

# Convert and execute
jupytext --to notebook --execute script.py

Cell organization rules:

• •# %% markers → New code cells
• •Multi-line strings at top → Markdown cells
• •Section comments → Markdown headers
• •Code blocks → Code cells

⚠️ IMPORTANT: Control cell boundaries for proper organization

By default, jupytext creates many small cells (splitting on blank lines). Two approaches to keep related code together:

Approach 1: Use # %% markers (simple)

Add # %% markers before each logical section:

# Header and imports (will be first cell)
import numpy as np

# %% Step 1: Load Data
data = pd.read_csv('data.csv')
print(data.head())

# %% Step 2: Preprocess
data_clean = data.dropna()

Approach 2: Custom script with existing section markers (recommended)

If your code already uses # ==== section markers, use the conversion script from this skill:

# Use the script from the skill's scripts/ directory
uv run python .shared/skills/learning-notebook_conversion/scripts/convert_to_notebook.py lab2_svm.py

This approach:

• •Uses existing code structure (no need to add # %% markers)
• •Keeps step headers together with their code
• •Creates cleaner notebooks with proper cell organization

Post-conversion tasks:

1. •

   Run all cells to generate outputs

2. •

   Replace plt.close() with plt.show() to display images inline:

   python

   import nbformat
   with open('notebook.ipynb', 'r', encoding='utf-8') as f:
       nb = nbformat.read(f, as_version=4)
   for cell in nb.cells:
       if cell.cell_type == 'code':
           cell.source = cell.source.replace('plt.close()', 'plt.show()')
   with open('notebook.ipynb', 'w', encoding='utf-8') as f:
       nbformat.write(nb, f)
   

   ⚠️ Do NOT use PowerShell for this - it will corrupt Chinese characters!

3. •

   Add markdown cells for better documentation:

   • •Title and metadata at top
   • •Section headers before major steps
   • •Explanations for complex logic
4. •

   Verify cell order:

   • •Logical flow maintained
   • •No circular dependencies
   • •Outputs display correctly
5. •

   Format markdown cells:

   markdown

   ## Step 1: Import Libraries
   
   Import required packages for data analysis.
   

4. Batch Conversion

Convert multiple files:

# All .ipynb to .py in directory
jupyter nbconvert --to python *.ipynb

# All .py to .ipynb
for file in *.py; do jupytext --to notebook "$file"; done

Python script for batch processing:

import os
from pathlib import Path
import nbformat
from nbconvert import PythonExporter

def convert_notebooks_to_py(directory):
    """Convert all .ipynb files in directory to .py"""
    for ipynb_file in Path(directory).glob('*.ipynb'):
        with open(ipynb_file, 'r', encoding='utf-8') as f:
            nb = nbformat.read(f, as_version=4)

        exporter = PythonExporter()
        source, _ = exporter.from_notebook_node(nb)

        py_file = ipynb_file.with_suffix('.py')
        with open(py_file, 'w', encoding='utf-8') as f:
            f.write(source)

        print(f"Converted: {ipynb_file} → {py_file}")

# Usage
convert_notebooks_to_py('labs/')

5. Conversion Best Practices

Before conversion:

• • Save and checkpoint current work
• • Run all cells (for .ipynb) to verify execution
• • Clear outputs if file size is large
• • Commit to version control

After conversion:

• • Test execution in target format
• • Verify all imports work
• • Check file paths are correct
• • Validate outputs match original
• • Update documentation if needed

Maintain consistency:

• •Keep both formats if needed for different use cases
• •Use version control to track changes
• •Document which format is "source of truth"
• •Sync changes between formats when updating

6. Format-Specific Considerations

When to use .ipynb:

• •Interactive exploration and visualization
• •Teaching and presentations
• •Sharing with outputs visible
• •Iterative development

When to use .py:

• •Version control (cleaner diffs)
• •Automated testing and CI/CD
• •Production deployment
• •Command-line execution
• •Code review

Hybrid approach:

• •Develop in .ipynb for interactivity
• •Convert to .py for submission/version control
• •Use jupytext to sync both formats automatically

Validation

After conversion, verify:

Code Execution

• • Converted file runs without errors
• • All imports resolve correctly
• • File paths work in new format
• • Outputs match original (if applicable)

Structure Preservation

• • Logical sections maintained
• • Comments and documentation intact
• • Function definitions complete
• • Variable scope correct

Academic Requirements

• • File-level docstring present (for .py)
• • Author and metadata included
• • Step order follows assignment
• • Submission format requirements met

Common Issues

Issue: Cell markers (# %%) not recognized

• •Solution: Install Python extension in VS Code
• •Or: Use Jupyter extension for interactive cells

Issue: Markdown cells lost in conversion

• •Solution: Use jupytext instead of basic nbconvert
• •Or: Manually convert important markdown to docstrings

Issue: Relative imports broken

• •Solution: Check working directory and adjust paths
• •Or: Use absolute imports with package structure

Issue: Magic commands cause errors in .py

• •Solution: Remove or comment out IPython magic commands:
  • •%matplotlib inline → Remove or comment
  • •!pip install → Convert to regular shell command
  • •%%time → Use time module instead

Issue: Outputs don't display in .py

• •Solution: Replace display() with print()
• •Add plt.show() for matplotlib plots
• •Use explicit print statements for results

Installation Requirements

# Basic conversion
pip install jupyter nbconvert

# Advanced conversion with sync
pip install jupytext

# For Python API
pip install nbformat

Example Workflow

User request: "把这个 notebook 转成 .py 文件"

Your workflow:

1. •Identify the .ipynb file (from context or ask user)
2. •Choose conversion method based on needs:
   • •Single file → VS Code export or nbconvert
   • •Batch → Script or command-line loop
   • •Custom processing → Python script
3. •Execute conversion
4. •Perform post-conversion cleanup:
   • •Add/verify docstrings
   • •Clean up cell markers
   • •Test execution
5. •Validate against checklist
6. •Inform user of completion and any issues

User request: "生成这个 .py 的 notebook 版本"

Your workflow:

1. •Identify the .py file
2. •Use jupytext to convert
3. •Open in Jupyter/VS Code
4. •Run all cells to generate outputs
5. •Add markdown cells for documentation
6. •Verify cell organization
7. •Save and inform user