Introduction to Statistics: What It Is, Why It Matters and Its Main Types

Every day we are surrounded by numbers: the price of petrol, the chance of rain this afternoon, the number of dengue cases in our district, or the CGPA on a transcript. Behind each of these numbers is statistics. In this post we will look at what statistics is, how it is used in real life, and the two main types of statistics that every beginner should know.

What is statistics?

Statistics is the science of collecting, organising, analysing, interpreting and presenting data. Its purpose is to help us draw conclusions and make decisions when we face uncertainty, that is, when we cannot observe everything or when the outcome is not yet known.

The word is used in two ways. As a discipline, statistics refers to the methods for working with data. As plural numbers, statistics are the figures produced from data, such as the unemployment rate, average household income or a course pass rate.

The statistical process in five steps:

1. Collect – gather data through surveys, experiments, sensors or existing records.

2. Organise – clean the data and arrange it in tables or spreadsheets.

3. Analyse – calculate summaries and fit statistical models.

4. Interpret – explain what the results mean for the question being asked.

5. Present – communicate the findings with charts, tables and reports.

Put simply, statistics turns data into information, and information into decisions.

The role of statistics in real life

Statistics is not just a subject in the classroom. It quietly supports decisions in almost every field. Here are some everyday examples:

• Health and medicine: Before a new drug is approved, clinical trials compare patients who receive the drug with those who receive a placebo. Statistical tests show whether the improvement is real or just due to chance. Health authorities also track weekly dengue and influenza cases to detect outbreaks early.

• Business and marketing: Companies analyse past sales to forecast demand and decide how much stock to order, and use customer surveys to measure satisfaction. An online shop may run an A/B test to see which website design leads to more purchases.

• Weather forecasting: When the forecast says there is a 70% chance of rain, it means that on past days with similar atmospheric conditions, rain fell about 7 times out of 10.

• Government and public policy: The population and housing census conducted by the Department of Statistics Malaysia (DOSM) tells the government how many people live in each area, along with their age, education and household characteristics. This guides where to build schools, clinics and roads.

• Education: Lecturers and teachers analyse exam results to find topics students struggle with, compare performance across classes, and set fair grade boundaries.

• Finance and insurance: Investors study the trend and volatility of gold prices or the stock market, while insurance companies use data on accidents and life expectancy to set premiums.

• Sports: Coaches use player statistics such as shooting percentage or distance covered to choose tactics and line-ups.

In each case, statistics helps people move from guessing to making evidence-based decisions.

Types of statistics

Statistics is commonly divided into two main branches: descriptive statistics and inferential statistics.

1. Descriptive statistics

Descriptive statistics summarise and describe the data you actually have. They do not try to go beyond the data. Common tools include:

• Measures of central tendency: mean, median and mode

• Measures of dispersion: range, variance, standard deviation and interquartile range

• Tables and graphs: frequency tables, bar charts, pie charts, histograms and box plots

Example: A lecturer reports that the average mark of 40 students in a class is 72, with a standard deviation of 8, and shows the distribution of marks in a histogram. This describes that particular class only.

2. Inferential statistics

Inferential statistics use data from a sample to draw conclusions about a larger population. Because we rarely have data on everyone, we study a smaller group and use probability to generalise, while stating how confident we are in the result. Common tools include:

• Estimation and confidence intervals

• Hypothesis testing, such as the t-test and chi-square test

• Correlation, regression and prediction models

Example: A survey of 1,000 randomly selected voters finds that 52% support a candidate. Using inferential statistics, we can estimate the support among all voters, for instance 52% ± 3% with 95% confidence.

Two key terms to remember: the population is the entire group we want to learn about, while the sample is the subset of the population that we actually observe. A good sample should be representative of the population, which is usually achieved through random sampling.

Summary

Statistics is the science of learning from data. It helps doctors test treatments, businesses plan ahead, governments allocate resources, and all of us make sense of the numbers we see every day. Descriptive statistics tell us what the data show; inferential statistics help us draw conclusions beyond the data we have. The sketchnote below summarises the key ideas of this article.

Feel free to save and share this sketchnote with your students and friends who are starting their statistics journey.

Feature Selection Explained: Filter, Wrapper & Embedded Methods for Machine Learning

More data is not always better. In many machine learning problems the real skill is not collecting every possible column, but choosing the handful that actually matter. Feature selection is the process of picking those useful clues and ignoring the noise. Do it well and your model trains faster, generalises better, and becomes far easier to interpret. Do it poorly and you waste compute, risk overfitting, and drown in meaningless signals.

Why feature selection matters

High-dimensional datasets introduce several problems:

• Curse of dimensionality — distance measures and model behaviour become unstable as the number of features grows.

• Overfitting — models learn noise and idiosyncrasies instead of general patterns.

• Longer training times and higher compute cost for both training and inference.

• Lower interpretability — more features makes it harder to explain predictions to stakeholders.

Example: imagine predicting diabetes. Glucose level or BMI are strong predictors. A patient ID or a postal code usually add nothing useful and can even confuse the model. The right selection transforms a sprawling dataset into a concentrated set of high-value clues.

The three main feature selection strategies

Feature selection techniques fall into three broad families. Each has strengths and trade-offs—pick the right one for your use case.

1. Filter methods — the quick background check

What they do: Rank features using statistical measures before any model is trained. Common scores include Pearson correlation, Chi-Square, mutual information and ANOVA F-values.

Why use them:

• Very fast and scalable to large datasets.

• Good for an initial pass to remove blatantly irrelevant features.

Limitations:

• Treat features independently, so they miss interactions and redundant combinations.

• May discard features that are weak alone but powerful together.

When to choose: Very large datasets or when you need a quick shortlist before deeper analysis.

2. Wrapper methods — the full investigation

What they do: Use a predictive model as a black box to evaluate different feature subsets. Examples include forward selection, backward elimination and recursive feature elimination (RFE).

Why use them:

• Consider feature interactions and often find combinations that produce the best predictive performance.

• Directly optimises for model performance on the chosen metric.

Limitations:

• Computationally expensive—testing many subsets means training many models.

• Prone to overfitting if feature subset search is not properly cross-validated.

When to choose: Smaller datasets or when maximum predictive performance matters and compute resources are available.

3. Embedded methods — the undercover operation

What they do: Perform selection during model training. Regularisation techniques like Lasso (L1) and Ridge (L2) are classic examples. Tree-based models also naturally provide feature importance signals.

Why use them:

• Efficient and often a good balance between performance and speed.

• Account for feature interactions as part of the learning process.

Limitations:

• Selection is tied to the specific model—features chosen by one algorithm might not be optimal for another.

When to choose: When you want automated, efficient selection integrated into model building—especially with regularised linear models or tree ensembles.

Feature selection vs dimensionality reduction

They are related but different:

• Feature selection picks a subset of the original features (keeps interpretability).

• Dimensionality reduction creates new features by transforming the original ones (e.g. PCA), trading interpretability for compactness.

Use selection when you need explainable models. Use reduction when you prioritise compression and the original features are less important.

Practical step-by-step workflow

1. Define the objective and evaluation metric (accuracy, AUC, RMSE, etc.).

2. Initial cleaning: remove IDs, constant features, and leak-prone columns.

3. Exploratory analysis: visualise distributions, spot missing values, and compute pairwise correlations.

4. Quick filter pass: use correlation, mutual information or Chi-Square to remove obviously irrelevant features.

5. Address multicollinearity: if features are highly correlated, keep the most meaningful or use PCA / domain knowledge to reduce redundancy.

6. Apply embedded or wrapper methods for fine-tuning. Use cross-validation inside the selection loop to avoid optimistic bias.

7. Validate final subset on a hold-out set and compare to the baseline (all features).

8. Document the decision: which features were removed and why, and the resulting performance change.

Decision guidance: which approach to pick?

• Huge dataset, limited compute: start with filter methods to shrink the feature set quickly.

• Small dataset, accuracy is critical: consider wrapper methods or RFE with strong cross-validation.

• Need a fast, well-balanced approach: use embedded methods (Lasso, tree-based feature importance).

Pitfalls and best practices

• Avoid data leakage. Run feature selection inside cross-validation; otherwise your performance estimate will be biased.

• Watch multicollinearity. Highly similar features can mislead models and importance measures.

• Scale and encode consistently. Normalisation and categorical encoding affect filter scores and regularisation behaviour.

• Blend domain knowledge with automated methods. Subject-matter insight often saves time and prevents removing subtle but important predictors.

• Measure cost versus benefit. Removing a feature that only marginally reduces error might still be valuable if it simplifies deployment or reduces latency.

Where feature selection really makes a difference

Applications are everywhere:

• Medicine: identify the few symptoms or biomarkers that predict disease.

• Finance: isolate signals that forecast market moves.

• Recommendation systems: focus on the interactions that drive user preferences.

• IoT and edge devices: pick only the sensor streams that matter to save bandwidth and power.

Final thought

In a world drowning in data, the skill that separates good models from great ones is often the ability to ignore the noise. Feature selection is the detective work of machine learning—find the real clues, toss the distractions, and build models that are faster, clearer and more reliable. What hidden pattern could you reveal if you started by saying no to everything except the evidence that matters?

Guide to install R and RStudio IDE into your PC

https://www.youtube.com/watch?v=3aXaw2w_ldM

Introduction

This guide shows a straightforward, step-by-step process to install R (the programming language) and RStudio (the integrated development environment) on your computer. Instructions cover Windows, macOS and Linux, and include quick checks to confirm a successful installation.

Before you begin (prerequisites)

• Internet connection to download installers.

• Know your operating system: Windows, macOS (Intel or Apple Silicon/M-series) or Linux.

• Administrator privileges to install applications on your machine.

• Recommended: 300 MB free space for RStudio and ~100 MB for R base (sizes vary by version).

Where to download

- R (The Comprehensive R Archive Network): https://cran.r-project.org

- RStudio Desktop: https://posit.co (RStudio downloads are available from the RStudio / Posit website)

Step 1 — Install R (base)

1. Open the CRAN website: https://cran.r-project.org

2. Choose your operating system: Windows, macOS or Linux.

3. Windows:

   1. Click "Download R for Windows" → "Install R for the first time".

   2. Download the current Windows installer (for example, R 4.5.2 as a recent example).

   3. Run the downloaded .exe, accept the prompts, choose English (or your preferred language), and follow the installer defaults.

4. macOS:

   1. Click "Download R for macOS".

   2. Pick the package that matches your processor: Apple Silicon/M-series or Intel.

   3. Open the downloaded .pkg and follow the installer steps.

5. Linux:

   1. Follow the appropriate link for your distribution on the CRAN pages and use the distribution-specific instructions (APT/YUM/other).

6. When installation finishes, you should find an R icon (for example, "R x.y.z") in your Start menu, Applications folder or desktop.

Step 2 — Install RStudio (IDE)

1. Open the RStudio (Posit) download page: https://posit.co (choose RStudio Desktop).

2. Make sure R (base) is already installed before installing RStudio. RStudio uses the installed R installation.

3. Choose the installer for your operating system:

   • Windows: download the RStudio Desktop .exe (example recent version: 2026.01.0+392).

   • macOS: download the .dmg for your macOS version.

   • Linux: select the appropriate package for your distribution.

4. Run the installer. On Windows you may be asked "Do you want to allow this app to make changes to your device?" — click Yes.

5. Choose the installation folder (default Program Files/C: is fine) and complete the installation.

6. When done, RStudio will usually place a desktop icon or an entry in your applications menu.

Verifying the installation

1. Open the R application to confirm R base runs. You should see an R console window and a prompt.

2. Open RStudio. The typical layout includes four panes: Console, Environment/History, Files/Plots/Packages/Help, and Source/Editor.

3. Check the R version inside R or RStudio console. Example commands: R.version.string sessionInfo()

4. Verify the R version matches the one you installed (for example, "R version 4.5.2").

Troubleshooting and tips

• If RStudio does not detect R, ensure R was installed in a default location and restart your computer. Reinstalling R first then RStudio often resolves detection problems.

• On macOS with Apple Silicon, install the correct package for M-series processors. Installing the Intel build on M-series may work under Rosetta but it is better to use the native build when available.

• If you see permission or UAC errors on Windows, run the installer as Administrator (right-click → Run as administrator).

• Keep both R and RStudio updated. R and RStudio versions are released independently; update R first, then RStudio.

• Need a specific R version for compatibility? CRAN archives old versions that can be downloaded if required.

Quick checklist before starting a project

• R base installed and version confirmed.

• RStudio installed and launches with the R console available.

• Install commonly used packages using install.packages("packageName") as needed.

Final notes

The installation process is simple: first install R base, then install RStudio Desktop. Following the installers' defaults will work for most users. Once installed, open RStudio and begin by checking your R version and installing any packages required for your work.

Peace and blessings. Assalamualaikum warahmatullahi wabarakatuh.