$$heading_1$$ Coursera上机器学习课程（公开课）汇总推荐

Coursera上有很多机器学习课程，这里做个总结，因为机器学习相关的概念和应用很多，这里推荐的课程仅限于和机器学习直接相关的课程，虽然深度学习属于机器学习范畴，这里暂时也将其排除在外，后续会专门推出深度学习课程的系列推荐。

1. Andrew Ng 老师的 机器学习课程（Machine Learning）

Machine learning is the study that allows computers to adaptively improve their performance with experience accumulated from the data observed. Our two sister courses teach the most fundamental algorithmic, theoretical and practical tools that any user of machine learning needs to know. This first course of the two would focus more on mathematical tools, and the other course would focus more on algorithmic tools. $$機器學習旨在讓電腦能由資料中累積的經驗來自我進步。我們的兩項姊妹課程將介紹各領域中的機器學習使用者都應該知道的基礎演算法、理論及實務工具。本課程將較為著重數學類的工具，而另一課程將較為著重方法類的工具。$$

Machine learning is the study that allows computers to adaptively improve their performance with experience accumulated from the data observed. Our two sister courses teach the most fundamental algorithmic, theoretical and practical tools that any user of machine learning needs to know. This second course of the two would focus more on algorithmic tools, and the other course would focus more on mathematical tools. $$機器學習旨在讓電腦能由資料中累積的經驗來自我進步。我們的兩項姊妹課程將介紹各領域中的機器學習使用者都應該知道的基礎演算法、理論及實務工具。本課程將較為著重方法類的工具，而另一課程將較為著重數學類的工具。 4. 华盛顿大学的 "机器学习专项课程（Machine Learning Specialization）" 这个系列课程包含4门子课程，分别是 机器学习基础：案例研究 , 机器学习：回归 , 机器学习：分类， 机器学习：聚类与检索: This Specialization from leading researchers at the University of Washington introduces you to the exciting, high-demand field of Machine Learning. Through a series of practical case studies, you will gain applied experience in major areas of Machine Learning including Prediction, Classification, Clustering, and Information Retrieval. You will learn to analyze large and complex datasets, create systems that adapt and improve over time, and build intelligent applications that can make predictions from data. 你是否好奇数据可以告诉你什么？你是否想在88集团赠送38彩金机器学习促进商业的核心方式上有深层次的理解？你是否想能同专家们讨论88集团赠送38彩金回归，分类，深度学习以及推荐系统的一切？在这门课上，你将会通过一系列实际案例学习来获取实践经历。 Do you have data and wonder what it can tell you? Do you need a deeper understanding of the core ways in which machine learning can improve your business? Do you want to be able to converse with specialists about anything from regression and classification to deep learning and recommender systems? In this course, you will get hands-on experience with machine learning from a series of practical case-studies. At the end of the first course you will have studied how to predict house prices based on house-level features, analyze sentiment from user reviews, retrieve documents of interest, recommend products, and search for images. Through hands-on practice with these use cases, you will be able to apply machine learning methods in a wide range of domains. This first course treats the machine learning method as a black box. Using this abstraction, you will focus on understanding tasks of interest, matching these tasks to machine learning tools, and assessing the quality of the output. In subsequent courses, you will delve into the components of this black box by examining models and algorithms. Together, these pieces form the machine learning pipeline, which you will use in developing intelligent applications. Learning Outcomes: By the end of this course, you will be able to: -Identify potential applications of machine learning in practice. -Describe the core differences in analyses enabled by regression, classification, and clustering. -Select the appropriate machine learning task for a potential application. -Apply regression, classification, clustering, retrieval, recommender systems, and deep learning. -Represent your data as features to serve as input to machine learning models. -Assess the model quality in terms of relevant error metrics for each task. -Utilize a dataset to fit a model to analyze new data. -Build an end-to-end application that uses machine learning at its core. -Implement these techniques in Python. 这门课程关注机器学习里面的一个基本问题: 回归（Regression)， 也通过案例研究（预测房价）的方式进行回归问题的学习，最终通过Python实现相关的机器学习算法。 Case Study - Predicting Housing Prices In our first case study, predicting house prices, you will create models that predict a continuous value [extract_itex$$price$$/extract_itex$$ from input features $$extract_itex$$square footage, number of bedrooms and bathrooms,...$$/extract_itex$$. This is just one of the many places where regression can be applied. Other applications range from predicting health outcomes in medicine, stock prices in finance, and power usage in high-performance computing, to analyzing which regulators are important for gene expression. In this course, you will explore regularized linear regression models for the task of prediction and feature selection. You will be able to handle very large sets of features and select between models of various complexity. You will also analyze the impact of aspects of your data -- such as outliers -- on your selected models and predictions. To fit these models, you will implement optimization algorithms that scale to large datasets. Learning Outcomes: By the end of this course, you will be able to: -Describe the input and output of a regression model. -Compare and contrast bias and variance when modeling data. -Estimate model parameters using optimization algorithms. -Tune parameters with cross validation. -Analyze the performance of the model. -Describe the notion of sparsity and how LASSO leads to sparse solutions. -Deploy methods to select between models. -Exploit the model to form predictions. -Build a regression model to predict prices using a housing dataset. -Implement these techniques in Python.

Case Studies: Analyzing Sentiment & Loan Default Prediction In our case study on analyzing sentiment, you will create models that predict a class $$extract_itex$$positive/negative sentiment$$/extract_itex$$ from input features $$extract_itex$$text of the reviews, user profile information,...$$/extract_itex$$. In our second case study for this course, loan default prediction, you will tackle financial data, and predict when a loan is likely to be risky or safe for the bank. These tasks are an examples of classification, one of the most widely used areas of machine learning, with a broad array of applications, including ad targeting, spam detection, medical diagnosis and image classification. In this course, you will create classifiers that provide state-of-the-art performance on a variety of tasks. You will become familiar with the most successful techniques, which are most widely used in practice, including logistic regression, decision trees and boosting. In addition, you will be able to design and implement the underlying algorithms that can learn these models at scale, using stochastic gradient ascent. You will implement these technique on real-world, large-scale machine learning tasks. You will also address significant tasks you will face in real-world applications of ML, including handling missing data and measuring precision and recall to evaluate a classifier. This course is hands-on, action-packed, and full of visualizations and illustrations of how these techniques will behave on real data. We've also included optional content in every module, covering advanced topics for those who want to go even deeper! Learning Objectives: By the end of this course, you will be able to: -Describe the input and output of a classification model. -Tackle both binary and multiclass classification problems. -Implement a logistic regression model for large-scale classification. -Create a non-linear model using decision trees. -Improve the performance of any model using boosting. -Scale your methods with stochastic gradient ascent. -Describe the underlying decision boundaries. -Build a classification model to predict sentiment in a product review dataset. -Analyze financial data to predict loan defaults. -Use techniques for handling missing data. -Evaluate your models using precision-recall metrics. -Implement these techniques in Python $$extract_itex$$or in the language of your choice, though Python is highly recommended$$/extract_itex$$.

Case Studies: Finding Similar Documents A reader is interested in a specific news article and you want to find similar articles to recommend. What is the right notion of similarity? Moreover, what if there are millions of other documents? Each time you want to a retrieve a new document, do you need to search through all other documents? How do you group similar documents together? How do you discover new, emerging topics that the documents cover? In this third case study, finding similar documents, you will examine similarity-based algorithms for retrieval. In this course, you will also examine structured representations for describing the documents in the corpus, including clustering and mixed membership models, such as latent Dirichlet allocation $$extract_itex$$LDA$$/extract_itex$$. You will implement expectation maximization $$extract_itex$$EM$$/extract_itex$$ to learn the document clusterings, and see how to scale the methods using MapReduce. Learning Outcomes: By the end of this course, you will be able to: -Create a document retrieval system using k-nearest neighbors. -Identify various similarity metrics for text data. -Reduce computations in k-nearest neighbor search by using KD-trees. -Produce approximate nearest neighbors using locality sensitive hashing. -Compare and contrast supervised and unsupervised learning tasks. -Cluster documents by topic using k-means. -Describe how to parallelize k-means using MapReduce. -Examine probabilistic clustering approaches using mixtures models. -Fit a mixture of Gaussian model using expectation maximization $$extract_itex$$EM$$/extract_itex$$. -Perform mixed membership modeling using latent Dirichlet allocation $$extract_itex$$LDA$$/extract_itex$$. -Describe the steps of a Gibbs sampler and how to use its output to draw inferences. -Compare and contrast initialization techniques for non-convex optimization objectives. -Implement these techniques in Python.

Python机器学习应用课程，这门课程主要聚焦在通过Python应用机器学习，包括机器学习和统计学的区别，机器学习工具包scikit-learn的介绍，有监督学习和无监督学习，数据泛化问题（例如交叉验证和过拟合）等。这门课程同时属于"Python数据科学应用专项课程系列（Applied Data Science with Python Specialization）"。

This course will introduce the learner to applied machine learning, focusing more on the techniques and methods than on the statistics behind these methods. The course will start with a discussion of how machine learning is different than descriptive statistics, and introduce the scikit learn toolkit. The issue of dimensionality of data will be discussed, and the task of clustering data, as well as evaluating those clusters, will be tackled. Supervised approaches for creating predictive models will be described, and learners will be able to apply the scikit learn predictive modelling methods while understanding process issues related to data generalizability $$extract_itex$$e.g. cross validation, overfitting$$/extract_itex$$. The course will end with a look at more advanced techniques, such as building ensembles, and practical limitations of predictive models. By the end of this course, students will be able to identify the difference between a supervised $$extract_itex$$classification$$/extract_itex$$ and unsupervised $$extract_itex$$clustering$$/extract_itex$$ technique, identify which technique they need to apply for a particular dataset and need, engineer features to meet that need, and write python code to carry out an analysis. This course should be taken after Introduction to Data Science in Python and Applied Plotting, Charting & Data Representation in Python and before Applied Text Mining in Python and Applied Social Analysis in Python.

6. 俄罗斯国立高等经济学院和Yandex联合推出的 高级机器学习专项课程系列（Advanced Machine Learning Specialization）

This specialization gives an introduction to deep learning, reinforcement learning, natural language understanding, computer vision and Bayesian methods. Top Kaggle machine learning practitioners and CERN scientists will share their experience of solving real-world problems and help you to fill the gaps between theory and practice. Upon completion of 7 courses you will be able to apply modern machine learning methods in enterprise and understand the caveats of real-world data and settings.

Bayesian methods are used in lots of fields: from game development to drug discovery. They give superpowers to many machine learning algorithms: handling missing data, extracting much more information from small datasets. Bayesian methods also allow us to estimate uncertainty in predictions, which is a really desirable feature for fields like medicine. When Bayesian methods are applied to deep learning, it turns out that they allow you to compress your models 100 folds, and automatically tune hyperparametrs, saving your time and money. In six weeks we will discuss the basics of Bayesian methods: from how to define a probabilistic model to how to make predictions from it. We will see how one can fully automate this workflow and how to speed it up using some advanced techniques. We will also see applications of Bayesian methods to deep learning and how to generate new images with it. We will see how new drugs that cure severe diseases be found with Bayesian methods.

7. 约翰霍普金斯大学的 Practical Machine Learning（机器学习实战）

One of the most common tasks performed by data scientists and data analysts are prediction and machine learning. This course will cover the basic components of building and applying prediction functions with an emphasis on practical applications. The course will provide basic grounding in concepts such as training and tests sets, overfitting, and error rates. The course will also introduce a range of model based and algorithmic machine learning methods including regression, classification trees, Naive Bayes, and random forests. The course will cover the complete process of building prediction functions including data collection, feature creation, algorithms, and evaluation.

This course focuses on one of the most important tools in your data analysis arsenal: regression analysis. Using either SAS or Python, you will begin with linear regression and then learn how to adapt when two variables do not present a clear linear relationship. You will examine multiple predictors of your outcome and be able to identify confounding variables, which can tell a more compelling story about your results. You will learn the assumptions underlying regression analysis, how to interpret regression coefficients, and how to use regression diagnostic plots and other tools to evaluate the quality of your regression model. Throughout the course, you will share with others the regression models you have developed and the stories they tell you.

Are you interested in predicting future outcomes using your data? This course helps you do just that! Machine learning is the process of developing, testing, and applying predictive algorithms to achieve this goal. Make sure to familiarize yourself with course 3 of this specialization before diving into these machine learning concepts. Building on Course 3, which introduces students to integral supervised machine learning concepts, this course will provide an overview of many additional concepts, techniques, and algorithms in machine learning, from basic classification to decision trees and clustering. By completing this course, you will learn how to apply, test, and interpret machine learning algorithms as alternative methods for addressing your research questions.

10. 加州大学圣地亚哥分校的 Machine Learning With Big Data（大数据机器学习）

Want to make sense of the volumes of data you have collected? Need to incorporate data-driven decisions into your process? This course provides an overview of machine learning techniques to explore, analyze, and leverage data. You will be introduced to tools and algorithms you can use to create machine learning models that learn from data, and to scale those models up to big data problems. At the end of the course, you will be able to: • Design an approach to leverage data using the steps in the machine learning process. • Apply machine learning techniques to explore and prepare data for modeling. • Identify the type of machine learning problem in order to apply the appropriate set of techniques. • Construct models that learn from data using widely available open source tools. • Analyze big data problems using scalable machine learning algorithms on Spark.

11. 俄罗斯搜索巨头Yandex推出的 Big Data Applications: Machine Learning at Scale（大数据应用：大规模机器学习）

Machine learning is transforming the world around us. To become successful, you’d better know what kinds of problems can be solved with machine learning, and how they can be solved. Don’t know where to start? The answer is one button away. During this course you will: - Identify practical problems which can be solved with machine learning - Build, tune and apply linear models with Spark MLLib - Understand methods of text processing - Fit decision trees and boost them with ensemble learning - Construct your own recommender system. As a practical assignment, you will - build and apply linear models for classification and regression tasks; - learn how to work with texts; - automatically construct decision trees and improve their performance with ensemble learning; - finally, you will build your own recommender system! With these skills, you will be able to tackle many practical machine learning tasks. We provide the tools, you choose the place of application to make this world of machines more intelligent.

$$heading_1$$ 斯坦福大学深度学习与自然语言处理第四讲：词窗口分类和神经网络

1. $$UFLDL tutorial$$
2. $$Learning Representations by Backpropogating Errors$$
3. 第四讲Slides $$slides$$
4. 第四讲视频 $$video$$

$$heading_1$$ PRML读书会第七章 Sparse Kernel Machines

PRML读书会第七章 Sparse Kernel Machines

（新浪微博: @豆角茄子麻酱凉面

$$heading_1$$ PRML读书会第四章 Linear Models for Classification

PRML读书会第四章 Linear Models for Classification

planktonli$$extract_itex$$1027753147$$/extract_itex$$ 19:52:28

1) Fisher准则的分类,以及它和最小二乘分类的关系 $$extract_itex$$Fisher分类是最小二乘分类的特例$$/extract_itex$$
2) 概率生成模型的分类模型
3) 概率判别模型的分类模型
4) 全贝叶斯概率的Laplace近似

1) 全贝叶斯
2) 经验贝叶斯
3) MAP贝叶斯

MAP$$extract_itex$$poor man’s Bayesian$$/extract_itex$$：不涉及marginalization，仅是一种按后验概率最大化的point estimate。这里的MAP$$extract_itex$$poor man’s Bayesian$$/extract_itex$$是属于 点概率估计的。而全贝叶斯可以看作对test样本的所有参数集合的加权平均，PRML说的Bayesian主要还是指Empirical Bayesian： 继续阅读

$$heading_1$$ 斯坦福大学机器学习第八课“神经网络的表示$$extract_itex$$Neural Networks: Representation$$/extract_itex$$”

1)  Non-linear hypotheses $$extract_itex$$非线性hypotheses$$/extract_itex$$

2)  Neurons and the brain $$extract_itex$$神经元和大脑$$/extract_itex$$

3)  Model representation I $$extract_itex$$模型表示一$$/extract_itex$$

4)  Model representation II $$extract_itex$$模型表示二$$/extract_itex$$

5)  Examples and intuitions I $$extract_itex$$例子和直观解释一$$/extract_itex$$

6)  Examples and intuitions II $$extract_itex$$例子和直观解释二$$/extract_itex$$

7)  Multi-class classification $$extract_itex$$多类分类问题$$/extract_itex$$

1)  Non-linear hypotheses $$extract_itex$$非线性hypotheses$$/extract_itex$$

2)  Neurons and the brain $$extract_itex$$神经元和大脑$$/extract_itex$$

• 起源于尝试让机器模仿大脑的算法；
• 在80年代和90年代早期非常流行，慢慢在90年代后期衰落；
• 最近得益于计算机硬件能力，又开始流行起来：对于很多应用，神经网络算法是一种“时髦”的技术；

3)  Model representation I $$extract_itex$$模型表示一$$/extract_itex$$

$$extract_itex$$a^{j}_i$$/extract_itex$$ = j层第i个单元的激活函数

$$extract_itex$$Theta^{$$extract_itex$$j$$/extract_itex$$}$$/extract_itex$$ = 从第j层映射到第j+1层的控制函数的权重矩阵

4)  Model representation II $$extract_itex$$模型表示二$$/extract_itex$$

5)  Examples and intuitions I $$extract_itex$$例子和直观解释一$$/extract_itex$$

6)  Examples and intuitions II $$extract_itex$$例子和直观解释二$$/extract_itex$$

7)  Multi-class classification $$extract_itex$$多类分类问题$$/extract_itex$$

http://en.wikipedia.org/wiki/Neural_network

http://en.wikipedia.org/wiki/Artificial_neural_network

$$heading_2$$ 神经网络编程入门

$$heading_3$$ 神经网络入门连载

http://library.thinkquest.org/29483/neural_index.shtml

http://home.agh.edu.pl/~vlsi/AI/xor_t/en/main.htm

http://en.wikipedia.org/wiki/NOR_logic

http://en.wikipedia.org/wiki/Logic_gate

$$heading_1$$ Coursera公开课笔记: 斯坦福大学机器学习第六课“逻辑回归$$extract_itex$$Logistic Regression$$/extract_itex$$”

1) Classification$$extract_itex$$分类$$/extract_itex$$

2) Hypothesis Representation

3) Decision boundary$$extract_itex$$决策边界$$/extract_itex$$

4) Cost function$$extract_itex$$代价函数，成本函数$$/extract_itex$$

5) Simplified cost function and gradient descent$$extract_itex$$简化版代价函数及梯度下降算法$$/extract_itex$$

6) Advanced optimization$$extract_itex$$其他优化算法$$/extract_itex$$

7) Multi-class classification: One-vs-all$$extract_itex$$多类分类问题$$/extract_itex$$

1) Classification$$extract_itex$$分类$$/extract_itex$$

1. 邮件：垃圾邮件/非垃圾邮件？
2. 在线交易：是否欺诈（是/否）？
3. 肿瘤：恶性/良性？

• 如果$$extract_itex$$h_theta$$extract_itex$$x$$/extract_itex$$ geq 0.5$$/extract_itex$$，则预测y=1,既y属于正例；
• 如果$$extract_itex$$h_theta$$extract_itex$$x$$/extract_itex$$ < 0.5$$/extract_itex$$，则预测y=0,既y属于负例；

2) Hypothesis Representation

Sigmoid 函数在有个很漂亮的“S"形，如下图所示（引自维基百科）：

Hypothesis输出的直观解释：

$$extract_itex$$h_theta$$extract_itex$$x$$/extract_itex$$$$/extract_itex$$ = 对于给定的输入x，y=1时估计的概率

3) Decision boundary$$extract_itex$$决策边界$$/extract_itex$$

$$extract_itex$$h_theta$$extract_itex$$x$$/extract_itex$$ < 0.5$$/extract_itex$$时，y = 0;

$$extract_itex$$g$$extract_itex$$z$$/extract_itex$$ geq 0.5$$/extract_itex$$时, $$extract_itex$$z geq 0$$/extract_itex$$;

$$extract_itex$$theta_0, theta_1, theta_2$$/extract_itex$$分别取-3, 1, 1,

4) Cost function$$extract_itex$$代价函数，成本函数$$/extract_itex$$

Hypothesis可表示为:

$$extract_itex$$h_theta$$extract_itex$$x$$/extract_itex$$ = frac{1}{1+e^{-theta^Tx}}$$/extract_itex$$

Cost Function:

$$extract_itex$$1$$/extract_itex$$ 0-1损失函数$$extract_itex$$0-1 loss function$$/extract_itex$$:

$$extract_itex$$2$$/extract_itex$$ 平方损失函数$$extract_itex$$quadratic loss function$$/extract_itex$$

$$extract_itex$$3$$/extract_itex$$ 绝对损失函数$$extract_itex$$absolute loss function$$/extract_itex$$

$$extract_itex$$4$$/extract_itex$$ 对数损失函数$$extract_itex$$logarithmic loss function$$/extract_itex$$ 或对数似然损失函数$$extract_itex$$log-likelihood loss function$$/extract_itex$$

5) Simplified cost function and gradient descent$$extract_itex$$简化版代价函数及梯度下降算法$$/extract_itex$$

???????x???????$$/extract_itex$$

h_theta$$extract_itex$$x$$/extract_itex$$

?????????????????????????" />$$extract_itex$$????????

?????????????????????????$$/extract_itex$$

theta$$extract_itex$$???$$/extract_itex$$J$$extract_itex$$theta$$/extract_itex$$

??????" />$$extract_itex$$:

??????$$/extract_itex$$

J$$extract_itex$$theta$$/extract_itex$$

?" />$$extract_itex$$????????????

?$$/extract_itex$$

J$$extract_itex$$theta$$/extract_itex$$

?????????????????????????????" />$$extract_itex$$?????????????

?????????????????????????????$$/extract_itex$$

h_theta$$extract_itex$$x$$/extract_itex$$$$extract_itex$$??????

6) Advanced optimization$$extract_itex$$??????$$/extract_itex$$

?????

????$$/extract_itex$$

theta

??????????????????

• Conjugate gradient method$$extract_itex$$?????$$/extract_itex$$
• Quasi-Newton method$$extract_itex$$????$$/extract_itex$$
• BFGS method
• L-BFGS$$extract_itex$$Limited-memory BFGS$$/extract_itex$$

????????????????????????????????

??????????????

??????????????

????????-???????????????????????????????????52nlp??????????????????????????????????????????????

???????????Quasi-Newton Method??LBFGS???????????????????????????????????????????????????????????????????????????????????????????
1) Numerical Methods for Unconstrained Optimization and Nonlinear Equations?J.E. Dennis Jr. Robert B. Schnabel?
2) Numerical Optimization?Jorge Nocedal Stephen J. Wright?

7) Multi-class classification: One-vs-all$$extract_itex$$??????$$/extract_itex$$

?????????

??????/??? ???????????????????

????$$extract_itex$$medical diagrams$$/extract_itex$$: ??????????

????????????

????????????

???????????

One-vs-all$$extract_itex$$one-vs-rest$$/extract_itex$$:

????????????????????????????????????????????????????

????????????????????

??-One-vs-all?????

?????? i ??????????????" />$$extract_itex$$?????????????

??????????????????

• Conjugate gradient method$$extract_itex$$?????$$/extract_itex$$
• Quasi-Newton method$$extract_itex$$????$$/extract_itex$$
• BFGS method
• L-BFGS$$extract_itex$$Limited-memory BFGS$$/extract_itex$$

????????????????????????????????

??????????????

??????????????

????????-???????????????????????????????????52nlp??????????????????????????????????????????????

???????????Quasi-Newton Method??LBFGS???????????????????????????????????????????????????????????????????????????????????????????
1) Numerical Methods for Unconstrained Optimization and Nonlinear Equations?J.E. Dennis Jr. Robert B. Schnabel?
2) Numerical Optimization?Jorge Nocedal Stephen J. Wright?

7) Multi-class classification: One-vs-all$$extract_itex$$??????$$/extract_itex$$

?????????

??????/??? ???????????????????

????$$extract_itex$$medical diagrams$$/extract_itex$$: ??????????

????????????

????????????

???????????

One-vs-all$$extract_itex$$one-vs-rest$$/extract_itex$$:

????????????????????????????????????????????????????

????????????????????

??-One-vs-all?????

?????? i ??????????????$$/extract_itex$$

h^{$$extract_itex$$i$$/extract_itex$$}_theta$$extract_itex$$x$$/extract_itex$$，并且预测 y = i时的概率；

http://en.wikipedia.org/wiki/Sigmoid_function

http://en.wikipedia.org/wiki/Logistic_function

http://en.wikipedia.org/wiki/Loss_function