동선생

In this post, I want to talk about the difference between GBM(Gradient Boosting Machine) and XGBoost.  Reading the previous post will be helpful for your understanding: [Kaggle Extra Study] 14. Tree-based Ensemble ModelsTree-based Ensemble Models?LightGBM, XGBoost, CatBoost, Random Forest are all tree-based ensemble models.These algorithms are popular on Kaggle because of their high prediction p..

Tree-based Ensemble Models?LightGBM, XGBoost, CatBoost, Random Forest are all tree-based ensemble models.These algorithms are popular on Kaggle because of their high prediction performance, fast learning speed, and ability to handle various data types.They show particularly powerful performance in analyzing structured data."Tree-based": These algorithms all use decision trees as their basic buil..

Weight Initialization?When training a neural network, one of the things we should care about is where we should start on the loss function to reach the optimal solution more easily and quickly.When training a neural network, the method of determining the starting position on the loss function is called Model Initialization.In particular, weights account for the largest portion of the model's par..

In Neural Networks, the activation function is a key element that enables the network to learn complex patterns and non-linearity.Reasons why activation functions are necessary:Introduction of Non-linearityWithout activation functions, neural networks would be merely combinations of linear transformations.No matter how many layers are stacked, they could be expressed as a single linear function,..

What is Drop-out?Drop-out is one of the methods to reduce overfitting in neural networks.Drop-out is not the only method to avoid overfitting.There are various methods including regularization, and while weight reduction methods like regularization are simple to implement and can avoid overfitting to some extent, when neural network models become complex, it becomes difficult to cope with overfi..

Birth of Logistic Loss FunctionIn linear regression, the goal was to find weights and bias that minimize the squared error between the actual and predicted values. Besides, the goal of classfication problems like logistic regression is to increase the actual ratio of correctly classified samples data itself. However, the ratio of correctly classified samples is not a differentiable function. In ..

Polars?Pandas library is widely used for data analysis and processing because it is flexible and convenient to use.However, Pandas has limitations in handling large-scale data due to processing speed and memory usage constraints when dealing with big data.To address this issues, Polars has emerged as a new data processing tool. Polars is implemented in Rust that offers great performance for proc..

Briefly About TabNetTabNet is a deep learning architecture developed by Google in 2019 for analyzing tabular data.In many competitions, including Kaggle, tree-based ensemble models like XGBoost and LightGBM usually dominate the leaderboards when it comes to tabular data analysis.While deep learning leads in analyzing unstructured data like images, text, and audio, tree-based methods have been wi..

From the previous post which was about overfitting and underfitting, we mentioned that one of the most popular methods to get rid of overfitting is to restrict the weights, which is called Regularization.  [Kaggle Study] 4. Overfitting, Underfitting, Variance and BiasOverfitting and UnderfittingOverfitting refers to cases where a model performs well on the training set but shows poor performance..

Overfitting and UnderfittingOverfitting refers to cases where a model performs well on the training set but shows poor performance on the validation set.Underfitting occurs when there isn't big difference between training and validation set performance, but both show poor performance.Learning CurvesLearning curve: Graph showing the model's learning progressLearning curve at left is a typical gra..

What is Learning Rate?Learning rate is a crucial hyperparameter in machine learning that controls the degree of updates on the model's weights.Generally, a loss function represents a hyperplane(초평면) in multi-dimensional space characterized by complex curvatures.If the weight updates are too large and overshoot the global minimum(the point where the loss function could be minimized), it becomes i..

During data preprocessing step of data analysis, it is necessary to check the scale of the features. What do you mean by "scale of feature"?It simply means the range of values that a particular feature possesses.For example, when A feature has 1 ~ 10 range of numerical values and B feature has 500 ~ 1000 range of numerical values, we can say that these 2 features has a large difference in scale...

Before we begin with the plots, you can find detailed information about data imputation from previous post: [Kaggle Extra Study] 7. Data ImputationWhile training machine learning models, it is necessary to preprocess missing values because it might greatly affect the performance of the model. Accordingly, there are various methods to deal with those missing values. Types of Missing Values:Missin..

We have to take different approach when dealing with time-series data. The fillna() method is used for imputing missing values in such problem.Basic Imputation Techniques:'ffill' or 'pad': Replace NaN values with last observed value'bfill' or 'backfill': Replace NaN values with next observed valueLinear Interpolation method1. Imputing using 'ffill' or 'pad'Code Example:city_day.fillna(method='ff..

#1 Detecting missing values# credit: https://www.kaggle.com/willkoehrsen/start-here-a-gentle-introduction. def missing_values_table(df): # Total missing values mis_val = df.isnull().sum() # Percentage of missing values mis_val_percent = 100 * df.isnull().sum() / len(df) # Make a table with the results mis_val_table = pd.concat([mis_val, m..