F-Principle
This is an exercise problem of the digital signal processing (DSP) course at
School of Artificial Intelligence at the Nanjing University (NJU),
teaching by Han-Jia Ye.
The course homepage is at DSP.
This exercise is written by Jia-Qi Yang.
Please feel free to contact me
by mailing [email protected] if you have any questions.
Problem 1: Understanding F-Principle (35pt)
Read following articles:
Then, answer following questions:
- What is F-Principle ? (5pt)
- Why F-Principle is important ? (5pt)
- What are the differences between response frequency and input frequency ? Which one is used in F-Principle ? (5pt)
- How is frequency defined in high-dimensional functions ? Why ? (10pt)
- How does the authors verify F-Principle experimentally ? (10pt)
Problem 2: Reproducing F-Principle (65pt)
Code to reproduce F-Principle by the authors is published at F-Principle Github.
You may modify F-Principle Github to conduct following experiments.
However, this implementation is based on tf1.x, and the high-dim experiments are not implemented.
You may also choose to extend pytorch training scripts provided in src/.
2.1 Low-dim Experiment (25pt)
Read F-Principle in low-dim experiments.
- Plot training procedure in Spatial Domain, i.e. the first figure in F-Principle in low-dim experiments. (10pt)
- Plot training procedure in Fourier Domain, i.e. the second figure in F-Principle in low-dim experiments. (10pt)
You may plot several figures instead of gifs in F-Principle in low-dim experiments.
2.2 High-dim Experiment (30pt)
Read F-Principle in high-dim experiments.
- Implement the projection method or the filtering method on MNIST dataset.
- Describe the procedure of your method using pseudo-code.
- Inspect how each response frequency component (e.g. high-frequency and low-frequency) converges.
You may plot figures or using tables to demostrate your results.
2.3 Summay (10pt)
- What did you learn from this practice problem ? (5pt)
- What problems did you encounter and how did you solve them ? (5pt)
