IGI Reading Group

The IGI Reading Group is the informal reading group a.k.a Journal Club of IGI where we meet about once a week and discuss papers that are related to our research, of general interest, or just some paper one of us found cool.

The journal club (usually) takes place on Fridays 14:00 in the former IGI Library.

If you’re from outside IGI and would like to attend a particular session, contact mueller [at] igi.tugraz.at.

Current presentation cycle

Anand, Arjun, Ceca, Christoph, Florian, Franz, Horst, Martin, Michael, Philipp, Samuel, Thomas L.

Upcoming meetings

Date	Moderator	Paper
30.07.2020	Arjun	Mittal, Sarthak, et al. “Learning to combine top-down and bottom-up signals in recurrent neural networks with attention over modules.” arXiv preprint arXiv:2006.16981 (2020).
		Goyal, Anirudh, et al. “Recurrent independent mechanisms.” arXiv preprint arXiv:1909.10893 (2019).
tba	tba

Wishlist

• Malherbe, Cédric, and Nicolas Vayatis. “Global optimization of Lipschitz functions.” arXiv preprint arXiv:1703.02628 (2017).
• Jin, Chi, et al. “How to escape saddle points efficiently.” arXiv preprint arXiv:1703.00887 (2017).
• IGI: Legenstein, Robert, Christian Naeger, and Wolfgang Maass. “What can a neuron learn with spike-timing-dependent plasticity?.” Neural computation 17.11 (2005): 2337-2382.
• IGI: Legenstein, Robert, and Wolfgang Maass. “On the classification capability of sign-constrained perceptrons.” Neural Computation 20.1 (2008): 288-309.
• Jolivet, Renaud, et al. “Predicting spike timing of neocortical pyramidal neurons by simple threshold models.” Journal of computational neuroscience 21.1 (2006): 35-49.
• IGI: Nessler, Bernhard, et al. “Bayesian computation emerges in generic cortical microcircuits through spike-timing-dependent plasticity.” PLoS Comput Biol 9.4 (2013): e1003037.
• Constantinople, Christine M., and Randy M. Bruno. “Deep cortical layers are activated directly by thalamus.” Science 340.6140 (2013): 1591-1594.
• LDA: Blei, David M., Andrew Y. Ng, and Michael I. Jordan. “Latent dirichlet allocation.” Journal of machine Learning research 3.Jan (2003): 993-1022.

Past meetings

Date	Moderator	Paper
02.03.2020	Luca	Hudson, Drew A., and Christopher D. Manning. “Compositional attention networks for machine reasoning.” arXiv preprint arXiv:1803.03067 (2018).
24.02.2020	Arjun	Schrittwieser, Julian, et al. “Mastering atari, go, chess and shogi by planning with a learned model.” arXiv preprint arXiv:1911.08265 (2019).
03.02.2020	Ceca	Introduction to ANOVA analysis (in Kass, Robert E., Uri T. Eden, and Emery N. Brown. Analysis of neural data. Vol. 491. New York: Springer, 2014.)
		Lindsay, Grace W., et al. “Hebbian learning in a random network captures selectivity properties of the prefrontal cortex.” Journal of Neuroscience 37.45 (2017): 11021-11036.
		Rigotti, Mattia, et al. “The importance of mixed selectivity in complex cognitive tasks.” Nature 497.7451 (2013): 585-590.
06.12.2019	Thomas	Henaff, Mikael, et al. “Tracking the world state with recurrent entity networks.” arXiv preprint arXiv:1612.03969 (2016).
02.12.2019	Philipp	Patel, Devdhar, et al. “Improved robustness of reinforcement learning policies upon conversion to spiking neuronal network platforms applied to ATARI games.” arXiv preprint arXiv:1903.11012 (2019).
25.11.2019	Special session
	Florian	Barrett, David GT, Sophie Deneve, and Christian K. Machens. “Optimal compensation for neuron loss.” Elife 5 (2016): e12454.
	Franz	Voelker, Aaron R., and Chris Eliasmith. “Improving spiking dynamical networks: Accurate delays, higher-order synapses, and time cells.” Neural computation 30.3 (2018): 569-609.
		Voelker, Aaron, Ivana Kajić, and Chris Eliasmith. “Legendre Memory Units: Continuous-Time Representation in Recurrent Neural Networks.” Advances in Neural Information Processing Systems. 2019.
	Arjun	Frady, E. Paxon, and Friedrich T. Sommer. “Robust computation with rhythmic spike patterns.” Proceedings of the National Academy of Sciences 116.36 (2019): 18050-18059.
11.11.2019	Michael	Habenschuss, Stefan, Zeno Jonke, and Wolfgang Maass. “Stochastic computations in cortical microcircuit models.” PLoS computational biology 9.11 (2013): e1003311.
		Berkes, Pietro, et al. “Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment.” Science 331.6013 (2011): 83-87.
29.10.2019	Darjan	Nayebi, Aran, et al. “Task-Driven convolutional recurrent models of the visual system.” Advances in Neural Information Processing Systems. 2018.
27.09.2019	Horst	Marblestone, Adam H., Greg Wayne, and Konrad P. Kording. “Toward an integration of deep learning and neuroscience.” Frontiers in computational neuroscience 10 (2016): 94.
07.06.2019	Franz	Hung, Chia-Chun, et al. “Optimizing agent behavior over long time scales by transporting value.” arXiv preprint arXiv:1810.06721 (2018)
16.05.2019	Elias	Frankle, Jonathan, and Michael Carbin. “The lottery ticket hypothesis: Finding sparse, trainable neural networks.” arXiv preprint arXiv:1803.03635 (2018).
09.05.2019	Rapid fire session
	Anand	Karnani, Mahesh M., et al. “A Blanket of Inhibition: Functional Inferences from Dense Inhibitory Connectivity”. Current Opinion in Neurobiology, 2014
		Okun, Michael, et al. “Diverse Coupling of Neurons to Populations in Sensory Cortex”. Nature, 2015.
	Darjan	Bönstrup, Marlene, et al. “A Rapid Form of Offline Consolidation in Skill Learning.” Current Biology (2019).
		Triefenbach, Fabian, et al. “Phoneme recognition with large hierarchical reservoirs.” Advances in neural information
	Michael	Saxe, Andrew M., et al. “On Random Weights and Unsupervised Feature Learning.” ICML. Vol. 2. No. 3. 2011.
	Philipp	Frady, Edward & Sommer, Friedrich. “Robust computation with rhythmic spike patterns.” arxiv preprint: arXiv:1901.07718 (2019)
	Arjun	Akrout, M., Wilson, C., Humphreys, P. C., Lillicrap, T., & Tweed, D. (2019). Using Weight Mirrors to Improve Feedback Alignment. arXiv preprint arXiv:1904.05391
	Ceca	Behrens, Timothy E. J., et al. “What Is a Cognitive Map? Organising Knowledge for Flexible Behaviour.” 2018, doi:10.1101/365593.; LINK: https://www.cell.com/neuron/pdf/S0896-6273(18)30856-0.pdf
18.04.2019	Ceca	O’Reilly, Randall C., Thomas E. Hazy, and Seth A. Herd. “The Leabra Cognitive Architecture: How to Play 20 Principles with Nature.​.” The Oxford handbook of cognitive science 91 (2016): 91-116.
11.04.2019	Darjan	Krotov, Dmitry, and John J. Hopfield. “Unsupervised learning by competing hidden units.” Proceedings of the National Academy of Sciences (2019): 201820458.
20.03.2019	Arjun	Koutnik, Jan, et al. “A clockwork rnn.” arXiv preprint arXiv:1402.3511 (2014).
		Chung, Junyoung, Sungjin Ahn, and Yoshua Bengio. “Hierarchical multiscale recurrent neural networks.” arXiv preprint arXiv:1609.01704 (2016).
14.03.2019	Anand	Jaderberg, Max, et al. “Human-level performance in first-person multiplayer games with population-based deep reinforcement learning.” arXiv preprint arXiv:1807.01281 (2018).
17.12.2018	Thomas L.	Beaulieu-Laroche, Lou, et al. “Enhanced Dendritic Compartmentalization in Human Cortical Neurons.” Cell 175.3 (2018): 643-651.
20.11.2018	Michael	Kutter, Esther F., et al. “Single Neurons in the Human Brain Encode Numbers.” Neuron (2018).
		Quiroga, R. Quian, et al. “Invariant visual representation by single neurons in the human brain.” Nature 435.7045 (2005): 1102.
09.11.2018	Guillaume	Wasmuht, Dante Francisco, et al. “Intrinsic neuronal dynamics predict distinct functional roles during working memory.” Nature communications 9.1 (2018): 3499.
19.10.2018	Franz	Perich, Matthew G., Juan A. Gallego, and Lee E. Miller. “A neural population mechanism for rapid learning.” Neuron (2018).
12.10.2018	Darjan	Zeng, Andy, et al. “Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning.” arXiv preprint arXiv:1803.09956 (2018).
		Dubey, Rachit, et al. “Investigating Human Priors for Playing Video Games.” arXiv preprint arXiv:1802.10217 (2018).
05.10.2018	Ceca	Rougier, Nicolas P., et al. “Prefrontal cortex and flexible cognitive control: Rules without symbols.” Proceedings of the National Academy of Sciences 102.20 (2005): 7338-7343.
21.09.2018	Arjun	Palm, Rasmus Berg, Ulrich Paquet, and Ole Winther. “Recurrent Relational Networks.” arXiv preprint arXiv:1711.08028 (2018).
10.08.2018	Anand	Franceschi, L., Frasconi, P., Salzo, S., Grazzi, R., & Pontil, M. (2018). Bilevel Programming for Hyperparameter Optimization and Meta-Learning. ArXiv:1806.04910 [Cs, Stat]. Retrieved from http://arxiv.org/abs/1806.04910 (ICML 2018)
03.08.2018	Darjan	Siwani, Samer, et al. “OLMα2 cells bidirectionally modulate learning.” Neuron (2018).
20.07.2018	Arjun	Henaff, Mikael, et al. “Tracking the world state with recurrent entity networks.” arXiv preprint arXiv:1612.03969 (2016).
13.07.2018	Franz	Sabour, Sara, Nicholas Frosst, and Geoffrey E. Hinton. “Dynamic routing between capsules.” Advances in Neural Information Processing Systems. 2017.
23.04.2018	Ceca	Glimcher, Paul W. “Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis.” Proceedings of the National Academy of Sciences 108.Supplement 3 (2011): 15647-15654.
12.03.2018	Franz	Houthooft, Rein, et al. “Vime: Variational information maximizing exploration.” Advances in Neural Information Processing Systems. 2016.
		Blundell, Charles, et al. “Weight uncertainty in neural networks.” arXiv preprint arXiv:1505.05424 (2015).
02.03.2018	Anand	Finn, Chelsea, Pieter Abbeel, and Sergey Levine. “Model-agnostic meta-learning for fast adaptation of deep networks.” arXiv preprint arXiv:1703.03400 (2017).
23.02.2018	Michael	Mostafa, Hesham, Vishwajith Ramesh, and Gert Cauwenberghs. “Deep supervised learning using local errors.” arXiv preprint arXiv:1711.06756 (2017).
09.02.2018	Anand	Costa, R., Assael, Y., Shillingford, B., de Freitas, N. & Vogels, Ti. Cortical microcircuits as gated-recurrent neural networks. in Advances in Neural Information Processing Systems 30 (eds. Guyon, I. et al.) 271–282 (Curran Associates, Inc., 2017).
02.02.2018	Guillaume	Bahdanau, Dzmitry, et al. “End-to-end attention-based large vocabulary speech recognition.” Acoustics, Speech and Signal Processing (ICASSP), 2016 IEEE International Conference on. IEEE, 2016.
		Graves, Alex, Abdel-rahman Mohamed, and Geoffrey Hinton. “Speech recognition with deep recurrent neural networks.” Acoustics, speech and signal processing (icassp), 2013 ieee international conference on. IEEE, 2013.
		Graves, Alex, et al. “Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks.” Proceedings of the 23rd international conference on Machine learning. ACM, 2006.
		Amodei, Dario, et al. “Deep speech 2: End-to-end speech recognition in english and mandarin.” International Conference on Machine Learning. 2016.
		Hannun, Awni, et al. “Deep speech: Scaling up end-to-end speech recognition.” arXiv preprint arXiv:1412.5567 (2014).
26.01.2018	Darjan	Mishra, Nikhil, et al. “Meta-learning with temporal convolutions.” arXiv preprint arXiv:1707.03141 (2017). https://arxiv.org/abs/1707.03141
19.01.2018	Arjun	Jaderberg, Max, et al. “Population Based Training of Neural Networks.” arXiv preprint arXiv:1711.09846 (2017).
12.01.2018	Thomas B.	Wang, Peng, et al. “Multi-attention network for one shot learning.” 2017 IEEE conference on computer vision and pattern recognition, CVPR. 2017.
05.01.2018	Thomas L.	Jaderberg, Max, et al. “Decoupled neural interfaces using synthetic gradients.” arXiv preprint arXiv:1608.05343 (2016).
07.12.2017	Franz	Graves. “Adaptive Computation Time for Recurrent Neural Networks.” arXiv:1603.08983 (2016). https://arxiv.org/abs/1603.08983
01.12.2017	Guillaume	Sussillo, Stavisky, Kao, Ryu, Shenoy. “Making brain-machine interfaces robust to future neural variability” nature communications
		Panzeri, Harvey, Piasini, Latham, Fellin “Cracking the Neural Code for Sensory Perception by Combining Statistics, Intervention and Behavior”
		Lee, Delbruck, and Pfeiffer “Training Deep Spiking Neural Networks Using Backpropagation”
24.11.2017	Anand	Xu, Yan, Xiaoqin Zeng, and Shuiming Zhong. “A new supervised learning algorithm for spiking neurons.” Neural computation 25.6 (2013): 1472-1511.
		Ponulak, Filip, and Andrzej Kasiński. “Supervised learning in spiking neural networks with ReSuMe: sequence learning, classification, and spike shifting.” Neural Computation 22.2 (2010): 467-510.
17.11.2017	Michael	Hadji, Isma, and Richard P. Wildes. “A Spatiotemporal Oriented Energy Network for Dynamic Texture Recognition.” arXiv preprint arXiv:1708.06690 (2017). https://arxiv.org/abs/1708.06690
06.10.2017	Guillaume	Song Han et al. 2017 - SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size https://arxiv.org/abs/1602.07360 (submitted to ICLR 2017)
		Song Han et al. 2017 - ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA https://arxiv.org/abs/1612.00694
		Collins et al. 2014 - Memory bounded neural network https://arxiv.org/pdf/1412.1442.pdf
		Song Han et al. 2015 - Learning both weights and connections https://arxiv.org/pdf/1506.02626.pdf (appeared in NIPS)
29.10.2017	Jian	Dvorkin R, Ziv NE (2016) Relative Contributions of Specific Activity Histories and Spontaneous Processes to Size Remodeling of Glutamatergic Synapses. PLoS Biol 14(10): e1002572. https://doi.org/10.1371/journal.pbio.1002572
		Rubinski A, Ziv NE (2015) Remodeling and Tenacity of Inhibitory Synapses: Relationships with Network Activity and Neighboring Excitatory Synapses. PLoS Comput Biol 11(11): e1004632. https://doi.org/10.1371/journal.pcbi.1004632
		Statman A, Kaufman M, Minerbi A, Ziv NE, Brenner N (2014) Synaptic Size Dynamics as an Effectively Stochastic Process. PLoS Comput Biol 10(10): e1003846. https://doi.org/10.1371/journal.pcbi.1003846
10.08.2017	Franz	Zoph, Barret, and Quoc V. Le. “Neural architecture search with reinforcement learning.” arXiv preprint arXiv:1611.01578 (2016).
02.08.2017	David	Friston K. and Kiebel S. “Predictive coding under the free-energy principle.” Phil. Trans. R. Soc. B (2009) 364, 1211–1221.
		Friston K. “Variational filtering.” NeuroImage (2008) 41, 747-766.
26.07.2017	Anand	Spratling, M. W. “A review of predictive coding algorithms.” Brain and cognition 112 (2017): 92-97.
		Rao, Rajesh PN, and Dana H. Ballard. “Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects.” Nature neuroscience 2.1 (1999): 79-87.
		PredNet: Lotter, William, Gabriel Kreiman, and David Cox. “Deep predictive coding networks for video prediction and unsupervised learning.” arXiv preprint arXiv:1605.08104 (2016).
26.07.2017	Michael M.	Dosovitskiy, Alexey, and Vladlen Koltun. “Learning to act by predicting the future.” arXiv preprint arXiv:1611.01779 (2016).
13.06.2017	Guillaume	Lillicrap, Timothy P., et al. “Random synaptic feedback weights support error backpropagation for deep learning.” Nature Communications 7 (2016).
25.04.2017	Guillaume	Salimans, Tim, et al. “Evolution Strategies as a Scalable Alternative to Reinforcement Learning.” arXiv preprint arXiv:1703.03864 (2017).
25.04.2017	Anand	Whittington, James CR, and Rafal Bogacz. “An Approximation of the Error Backpropagation Algorithm in a Predictive Coding Network with Local Hebbian Synaptic Plasticity.” Neural Computation (2017).
25.04.2017	Arjun	Orhan, A. Emin, and Wei Ji Ma. “Efficient Probabilistic Inference in Generic Neural Networks Trained with Non-Probabilistic Feedback.” arXiv preprint arXiv:1601.03060 (2016). APA
25.04.2017	David	Schiess, Mathieu, Robert Urbanczik, and Walter Senn. “Somato-dendritic synaptic plasticity and error-backpropagation in active dendrites.” PLoS Comput Biol 12.2 (2016): e1004638.
18.04.2017	Anand	Goodfellow, Ian, et al. “Generative adversarial nets.” Advances in neural information processing systems. 2014.
14.04.2017	David	Variational Auto-encoders
		Kingma, Diederik P., and Max Welling. “Auto-encoding variational bayes.” arXiv preprint arXiv:1312.6114 (2013).
04.04.2017	Guillaume and David	Variational Inference
		Mnih, Andriy, and Karol Gregor. “Neural variational inference and learning in belief networks.” arXiv preprint arXiv:1402.0030 (2014).
28.03.2017	Arjun	Dirichlet Distributions
		Blei, David M., and Michael I. Jordan. “Variational inference for Dirichlet process mixtures.” Bayesian analysis 1.1 (2006): 121-143.
		Sethuraman, Jayaram. “A constructive definition of Dirichlet priors.” Statistica sinica (1994): 639-650.
		Blackwell, David, and James B. MacQueen. “Ferguson distributions via Pólya urn schemes.” The annals of statistics (1973): 353-355.
		Ferguson, Thomas S. “A Bayesian analysis of some nonparametric problems.” The annals of statistics (1973): 209-230.
02.12.2016	Arjun	Nessler, Bernhard, et al. “Bayesian computation emerges in generic cortical microcircuits through spike-timing-dependent plasticity.” PLoS Comput Biol 9.4 (2013): e1003037.
18.11.2016	Guillaume	Nithianantharajah, Jess, et al. “Synaptic scaffold evolution generated components of vertebrate cognitive complexity.” Nature neuroscience 16.1 (2013): 16-24.
		Carlisle, Holly J., et al. “Opposing effects of PSD‐93 and PSD‐95 on long‐term potentiation and spike timing‐dependent plasticity.” The Journal of physiology 586.24 (2008): 5885-5900.
11.11.2016	Anand	Rigotti, Mattia, et al. “The importance of mixed selectivity in complex cognitive tasks.” Nature 497.7451 (2013): 585-590.
09.09.2016	Ke Bai	Eliasmith, Chris, et al. “A large-scale model of the functioning brain.” science 338.6111 (2012): 1202-1205.
02.09.2016	Ke Bai	Bobier, Bruce, Terrence C. Stewart, and Chris Eliasmith. “A unifying mechanistic model of selective attention in spiking neurons.” PLoS Comput Biol 10.6 (2014): e1003577.
16.08.2016	David	Zenke, Friedemann, Everton J. Agnes, and Wulfram Gerstner. “Diverse synaptic plasticity mechanisms orchestrated to form and retrieve memories in spiking neural networks.” Nature communications 6 (2015).
05.08.2016	Zhaofei	Raju, Rajkumar Vasudeva, and Xaq Pitkow. “Inference by Reparameterization in Neural Population Codes.” Advances in Neural Information Processing Systems. 2016.
28.07.2016	Anna	Buzsáki, György. “Neural syntax: cell assemblies, synapsembles, and readers.” Neuron 68.3 (2010): 362-385.
21.07.2016	Guillaume	Chung, Junyoung, et al. “Empirical evaluation of gated recurrent neural networks on sequence modeling.” arXiv preprint arXiv:1412.3555 (2014).
		Sussillo, David, and L. F. Abbott. “Random walk initialization for training very deep feedforward networks.” arXiv preprint arXiv:1412.6558 (2014).
27.05.2016	Guillaume	Williams, Ronald J. “Simple statistical gradient-following algorithms for connectionist reinforcement learning.” Machine learning 8.3-4 (1992): 229-256.
24.03.2016	Anand	Denève, Sophie, and Christian K. Machens. “Efficient codes and balanced networks.” Nature neuroscience 19.3 (2016): 375-382.
17.03.2016	Anand	Abbott, L. F., Brian DePasquale, and Raoul-Martin Memmesheimer. “Building functional networks of spiking model neurons.” Nature neuroscience 19.3 (2016): 350-355.
10.03.2016	David	Hochreiter, Sepp, and Jürgen Schmidhuber. “Long short-term memory.” Neural computation 9.8 (1997): 1735-1780.
		Graves, Alex, and Jürgen Schmidhuber. “Offline handwriting recognition with multidimensional recurrent neural networks.” Advances in neural information processing systems. 2009.
		Graves, Alex. “Generating sequences with recurrent neural networks.” arXiv preprint arXiv:1308.0850 (2013).
		Graves, Alex, Greg Wayne, and Ivo Danihelka. “Neural turing machines.” arXiv preprint arXiv:1410.5401 (2014).
26.02.2016	Guillaume	Gardner, Brian, Ioana Sporea, and André Grüning. “Learning spatiotemporally encoded pattern transformations in structured spiking neural networks.” Neural computation (2015).
15.12.2015	Guillaume	Hennequin, Guillaume, Tim P. Vogels, and Wulfram Gerstner. “Optimal control of transient dynamics in balanced networks supports generation of complex movements.” Neuron 82.6 (2014): 1394-1406.
11.12.2015	Gernot	Avermann, Michael, et al. “Microcircuits of excitatory and inhibitory neurons in layer 2/3 of mouse barrel cortex.” Journal of neurophysiology 107.11 (2012): 3116-3134.
31.11.2015	Christoph	Mante, Valerio, et al. “Context-dependent computation by recurrent dynamics in prefrontal cortex.” Nature 503.7474 (2013): 78-84.
17.11.2015	David	Pfister, Jean-Pascal, Peter Dayan, and Máté Lengyel. “Synapses with short-term plasticity are optimal estimators of presynaptic membrane potentials.” Nature neuroscience 13.10 (2010): 1271-1275.
27.10.2015	Zhaofei	Habenschuss, Stefan, Helmut Puhr, and Wolfgang Maass. “Emergence of optimal decoding of population codes through STDP.” Neural computation 25.6 (2013): 1371-1407.
20.10.2015	Anand	Maass, Wolfgang, Thomas Natschläger, and Henry Markram. “Real-time computing without stable states: A new framework for neural computation based on perturbations.” Neural computation 14.11 (2002): 2531-2560.
13.10.2015	Guillaume	Brunel, Nicolas. “Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons.” Journal of computational neuroscience 8.3 (2000): 183-208.