--- title: K-Fold Cross-Validation in Scikit-Learn: Tutorial | daily.dev description: Learn how K-Fold Cross-Validation improves machine learning models by providing reliable performance estimates and preventing overfitting. canonical: https://daily.dev/blog/k-fold-cross-validation-in-scikit-learn-tutorial/ og:type: article og:url: https://daily.dev/blog/k-fold-cross-validation-in-scikit-learn-tutorial/ og:title: K-Fold Cross-Validation in Scikit-Learn: Tutorial | daily.dev og:description: Learn how K-Fold Cross-Validation improves machine learning models by providing reliable performance estimates and preventing overfitting. og:image: https://media.daily.dev/image/upload/s--4tRLj3jE--/f_auto,q_auto/v1/recruiter-landing/66cfbb2d2687c47441f88f6c_ddc49126cb4364de9fd8ba4ecd6269ba_56718896f6?_a=BAMAMiB80 og:site_name: daily.dev og:locale: en_US article:published_time: 2024-08-29 article:modified_time: 2026-05-25T06:19:43.982Z article:author: Alex Carter twitter:card: summary_large_image twitter:site: @dailydotdev twitter:creator: @dailydotdev twitter:title: K-Fold Cross-Validation in Scikit-Learn: Tutorial | daily.dev twitter:description: Learn how K-Fold Cross-Validation improves machine learning models by providing reliable performance estimates and preventing overfitting. twitter:image: https://media.daily.dev/image/upload/s--4tRLj3jE--/f_auto,q_auto/v1/recruiter-landing/66cfbb2d2687c47441f88f6c_ddc49126cb4364de9fd8ba4ecd6269ba_56718896f6?_a=BAMAMiB80 --- K-Fold Cross-Validation helps you build better machine learning models. Here's what you need to know: - Splits data into K parts for training and testing - Uses all data for both training and testing - Gives more reliable performance estimates - Helps prevent overfitting Key steps: 1. Pick number of folds (K) 2. Split data into K equal parts 3. Train on K-1 parts, test on 1 part 4. Repeat K times 5. Average the results [Scikit-Learn](https://scikit-learn.org/) code: ```python from sklearn.model_selection import KFold kf = KFold(n_splits=5, shuffle=True, random_state=42) for train_index, test_index in kf.split(X): X_train, X_test = X[train_index], X[test_index] y_train, y_test = y[train_index], y[test_index] # Train and evaluate model here ``` Quick comparison: | Method | Pros | Cons | | --- | --- | --- | | K-Fold CV | Uses all data, reduces bias | More computationally expensive | | Simple Split | Fast, easy | Less reliable estimates | | LOOCV | Low bias | Very computationally expensive | K-Fold Cross-Validation helps you build more reliable models by giving a fuller picture of performance. ## Related video from YouTube ::: @iframe https://www.youtube-nocookie.com/embed/1AGuK_8LkGQ ## What You Need to Know First Before diving in, make sure you have: 1. [Python](https://www.python.org/) (3.5+) installed 2. Scikit-Learn installed: - Via pip: `pip install -U scikit-learn` - Via conda: `conda install scikit-learn` Key libraries: | Library | Min Version | | --- | --- | | [NumPy](https://numpy.org/) | 1.11.0 | | [SciPy](https://scipy.org/) | 0.17.0 | | [Joblib](https://joblib.readthedocs.io/) | 0.11 | | [Matplotlib](https://matplotlib.org/) | 1.5.1 | | [Pandas](https://pandas.pydata.org/) | 0.18.0 | Understand these concepts: - Model training and testing - Overfitting and underfitting - Model evaluation metrics ## What is K-Fold Cross-Validation? K-Fold Cross-Validation assesses model performance on new data. It works like this: 1. Split data into K equal parts 2. Train on K-1 parts, test on 1 part 3. Repeat K times 4. Average the results Example with 5-fold: | Iteration | Training Folds | Testing Fold | | --- | --- | --- | | 1 | 2, 3, 4, 5 | 1 | | 2 | 1, 3, 4, 5 | 2 | | 3 | 1, 2, 4, 5 | 3 | | 4 | 1, 2, 3, 5 | 4 | | 5 | 1, 2, 3, 4 | 5 | Why it's better: 1. Uses more data for training 2. Reduces bias 3. Gives better performance estimates 4. Allows confidence interval calculation > "Cross-validation can detect overfitting, showing if a model isn't generalizing well to new data." - Nisha Arya, Data Scientist ## Getting Ready to Code Set up your environment: ```python import numpy as np import pandas as pd from sklearn.model_selection import KFold from sklearn.datasets import load_iris, load_diabetes # Load datasets iris = load_iris() X_iris, y_iris = iris.data, iris.target diabetes = load_diabetes() X_diabetes, y_diabetes = diabetes.data, diabetes.target print("Iris dataset shape:", X_iris.shape) print("Diabetes dataset shape:", X_diabetes.shape) ``` For real-world data: ```python url = "https://archive.ics.uci.edu/ml/machine-learning-databases/00267/data_banknote_authentication.txt" banknote_df = pd.read_csv(url, header=None, names=['variance', 'skewness', 'curtosis', 'entropy', 'class']) ``` ###### sbb-itb-bfaad5b ## Using K-Fold Cross-Validation in [Scikit-Learn](https://scikit-learn.org/)  Basic setup: ```python from sklearn.model_selection import KFold kf = KFold(n_splits=5, shuffle=True, random_state=42) for train_index, test_index in kf.split(X): X_train, X_test = X[train_index], X[test_index] y_train, y_test = y[train_index], y[test_index] # Train and evaluate model here ``` With a model: ```python from sklearn.linear_model import LogisticRegression from sklearn.model_selection import cross_validate log_reg = LogisticRegression(solver='liblinear') cv_results = cross_validate(log_reg, X_iris, y_iris, cv=kf, scoring='accuracy') print("Cross-validation scores:", cv_results['test_score']) print("Mean accuracy:", cv_results['test_score'].mean()) ``` ## Changing K-Fold Settings Adjust folds: ```python kf = KFold(n_splits=10, shuffle=True, random_state=42) ``` Use Stratified K-Fold: ```python from sklearn.model_selection import StratifiedKFold skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42) ``` ## Checking How Well Your Model Works Get scores: ```python from sklearn.model_selection import cross_val_score from sklearn import svm clf = svm.SVC(kernel='linear', C=1) scores = cross_val_score(clf, X, y, cv=5) print(f"Scores: {scores}") print(f"Mean: {scores.mean():.2f}") print(f"Standard Deviation: {scores.std():.2f}") ``` Use different metrics: ```python scores = cross_val_score(clf, X, y, cv=5, scoring='f1') ``` ## Tips and Common Mistakes 1. Pick 5-10 folds for most datasets 2. Use Stratified K-Fold for uneven data 3. Prevent data leakage: - Split before preprocessing - Use Scikit-Learn's Pipeline ```python from sklearn.pipeline import Pipeline from sklearn.preprocessing import StandardScaler from sklearn.svm import SVC pipeline = Pipeline([ ('scaler', StandardScaler()), ('svm', SVC()) ]) ``` K-Fold Cross-Validation helps build reliable models by providing fuller performance estimates and preventing overfitting. ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/og-image.png?v=a830cdf1","width":1200,"height":630},"sameAs":["https://twitter.com/dailydotdev","https://www.linkedin.com/company/dailydotdev","https://github.com/dailydotdev","https://www.instagram.com/dailydotdev"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","description":"Free, personalized developer news aggregator. Stay on top of software development news, AI coding tools, and web dev - curated daily from trusted sources.","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}},{"@type":"WebPage","@id":"https://daily.dev/blog/k-fold-cross-validation-in-scikit-learn-tutorial/","url":"https://daily.dev/blog/k-fold-cross-validation-in-scikit-learn-tutorial/","name":"K-Fold Cross-Validation in Scikit-Learn: Tutorial | daily.dev","description":"Learn how K-Fold Cross-Validation improves machine learning models by providing reliable performance estimates and preventing overfitting.","inLanguage":"en-US","isPartOf":{"@id":"https://daily.dev/#website"},"timeRequired":"PT3M"},{"@type":"Article","@id":"https://daily.dev/blog/k-fold-cross-validation-in-scikit-learn-tutorial/#article","headline":"K-Fold Cross-Validation in Scikit-Learn: Tutorial","url":"https://daily.dev/blog/k-fold-cross-validation-in-scikit-learn-tutorial/","datePublished":"2024-08-29","dateModified":"2026-05-25T06:19:43.982Z","isPartOf":{"@id":"https://daily.dev/#website"},"publisher":{"@id":"https://daily.dev/#organization"},"mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/blog/k-fold-cross-validation-in-scikit-learn-tutorial/"},"description":"Learn how K-Fold Cross-Validation improves machine learning models by providing reliable performance estimates and preventing overfitting.","image":{"@type":"ImageObject","url":"https://media.daily.dev/image/upload/s--4tRLj3jE--/f_auto,q_auto/v1/recruiter-landing/66cfbb2d2687c47441f88f6c_ddc49126cb4364de9fd8ba4ecd6269ba_56718896f6?_a=BAMAMiB80"},"author":{"@type":"Person","name":"Alex Carter","url":"https://app.daily.dev/alexcarterdev"},"timeRequired":"PT3M","potentialAction":{"@type":"ReadAction","target":"https://daily.dev/blog/k-fold-cross-validation-in-scikit-learn-tutorial/"}},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://daily.dev/blog/"},{"@type":"ListItem","position":3,"name":"AI","item":"https://daily.dev/categories/ai/"},{"@type":"ListItem","position":4,"name":"K-Fold Cross-Validation in Scikit-Learn: Tutorial","item":"https://daily.dev/blog/k-fold-cross-validation-in-scikit-learn-tutorial/"}]}]} ```