Fundamentos de Machine Learning: Supervised, Unsupervised y Reinforcement Learning con Algoritmos
Este artículo es una introducción exhaustiva a los fundamentos de Machine Learning, incluyendo Supervised, Unsupervised y Reinforcement Learning con algoritmos y ejemplos prácticos.
En Resumen
Machine Learning permite a las computadoras aprender a partir de datos. Supervised Learning aprende con datos etiquetados, Unsupervised Learning encuentra patrones en datos sin etiquetar, y Reinforcement Learning aprende mediante recompensas.
Descripción Técnica Compacta
Machine Learning es un área de la inteligencia artificial donde los algoritmos aprenden de los datos sin necesidad de ser programados explícitamente.
Categorías de aprendizaje:
Supervised Learning
- Concepto: Aprendizaje con datos de entrenamiento etiquetados
- Objetivo: Realizar predicciones sobre datos nuevos no vistos
- Tipos: Clasificación (valores discretos), Regresión (valores continuos)
- Algoritmos: Linear Regression, Decision Trees, Random Forest, SVM, Neural Networks
Unsupervised Learning
- Concepto: Aprendizaje sin datos etiquetados
- Objetivo: Descubrir estructuras y patrones en los datos
- Tipos: Clustering, Dimensionality Reduction, Association
- Algoritmos: K-Means, Hierarchical Clustering, PCA, Apriori
Reinforcement Learning
- Concepto: Aprendizaje mediante interacción con el entorno
- Objetivo: Maximizar la recompensa acumulativa
- Tipos: Model-based, Model-free, Multi-agent
- Algoritmos: Q-Learning, Deep Q-Networks, Policy Gradients
Puntos Clave de Referencia
- Machine Learning: Aprendizaje automático a partir de datos
- Supervised Learning: Aprendizaje con datos etiquetados (Clasificación, Regresión)
- Unsupervised Learning: Aprendizaje sin etiquetas (Clustering, Reconocimiento de patrones)
- Reinforcement Learning: Aprendizaje mediante recompensas (Agente, Entorno, Acciones)
- Training/Testing: División de datos para validación de modelos
- Overfitting/Underfitting: Problemas de ajuste de modelos
- Feature Engineering: Preparación y transformación de datos
- Relevancia tecnológica: Tecnologías de IA modernas y aplicaciones
Componentes Principales
- Datos: Datos de entrenamiento, validación y prueba
- Features: Variables de entrada y características
- Modelos: Funciones matemáticas y algoritmos
- Entrenamiento: Ajuste de parámetros del modelo
- Evaluación: Medición de rendimiento y validación
- Predicción: Predicciones para datos nuevos
- Optimización: Ajuste de hiperparámetros
- Despliegue: Integración en sistemas de producción
Ejemplos Prácticos
1. Supervised Learning con Python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression, LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score, mean_squared_error, classification_report
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import make_classification, make_regression
# Supervised Learning Demo
class SupervisedLearningDemo:
def __init__(self):
self.models = {}
self.results = {}
# Lineare Regression
def linear_regression_demo(self):
print("=== Lineare Regression Demo ===")
# Synthetische Daten erstellen
np.random.seed(42)
X, y = make_regression(n_samples=100, n_features=1, noise=10, random_state=42)
# Daten aufteilen
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Modell trainieren
model = LinearRegression()
model.fit(X_train, y_train)
# Vorhersagen
y_train_pred = model.predict(X_train)
y_test_pred = model.predict(X_test)
# Evaluation
train_mse = mean_squared_error(y_train, y_train_pred)
test_mse = mean_squared_error(y_test, y_test_pred)
print(f"Training MSE: {train_mse:.2f}")
print(f"Test MSE: {test_mse:.2f}")
print(f"Koeffizient: {model.coef_[0]:.2f}")
print(f"Intercept: {model.intercept_:.2f}")
# Ergebnisse speichern
self.models['linear_regression'] = model
self.results['linear_regression'] = {
'train_mse': train_mse,
'test_mse': test_mse,
'r2_score': model.score(X_test, y_test)
}
return X_train, X_test, y_train, y_test, y_test_pred
# Logistische Regression (Klassifikation)
def logistic_regression_demo(self):
print("\n=== Logistische Regression Demo ===")
# Klassifikationsdaten erstellen
X, y = make_classification(n_samples=200, n_features=2, n_redundant=0,
n_informative=2, random_state=42, n_clusters_per_class=1)
# Daten aufteilen
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Features skalieren
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)
# Modell trainieren
model = LogisticRegression(random_state=42)
model.fit(X_train_scaled, y_train)
# Vorhersagen
y_train_pred = model.predict(X_train_scaled)
y_test_pred = model.predict(X_test_scaled)
# Evaluation
train_accuracy = accuracy_score(y_train, y_train_pred)
test_accuracy = accuracy_score(y_test, y_test_pred)
print(f"Training Accuracy: {train_accuracy:.3f}")
print(f"Test Accuracy: {test_accuracy:.3f}")
print("Test Classification Report:")
print(classification_report(y_test, y_test_pred))
# Ergebnisse speichern
self.models['logistic_regression'] = model
self.results['logistic_regression'] = {
'train_accuracy': train_accuracy,
'test_accuracy': test_accuracy
}
return X_train_scaled, X_test_scaled, y_train, y_test, y_test_pred
# Decision Tree Classifier
def decision_tree_demo(self):
print("\n=== Decision Tree Demo ===")
# Komplexere Klassifikationsdaten
X, y = make_classification(n_samples=300, n_features=4, n_redundant=1,
n_informative=3, random_state=42, n_clusters_per_class=2)
# Daten aufteilen
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Decision Tree mit verschiedenen Tiefen
depths = [3, 5, 10, None]
for depth in depths:
model = DecisionTreeClassifier(max_depth=depth, random_state=42)
model.fit(X_train, y_train)
# Vorhersagen
y_train_pred = model.predict(X_train)
y_test_pred = model.predict(X_test)
# Evaluation
train_accuracy = accuracy_score(y_train, y_train_pred)
test_accuracy = accuracy_score(y_test, y_test_pred)
print(f"Max Depth {depth if depth else 'None'}:")
print(f" Training Accuracy: {train_accuracy:.3f}")
print(f" Test Accuracy: {test_accuracy:.3f}")
# Overfitting erkennen
overfitting = train_accuracy - test_accuracy
if overfitting > 0.1:
print(f" ⚠️ Overfitting detected (diff: {overfitting:.3f})")
# Bestes Modell speichern
best_model = DecisionTreeClassifier(max_depth=5, random_state=42)
best_model.fit(X_train, y_train)
self.models['decision_tree'] = best_model
return X_train, X_test, y_train, y_test
# Random Forest
def random_forest_demo(self):
print("\n=== Random Forest Demo ===")
# Hochdimensionale Daten
X, y = make_classification(n_samples=500, n_features=10, n_redundant=3,
n_informative=7, random_state=42, n_clusters_per_class=2)
# Daten aufteilen
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Random Forest mit verschiedenen Baumanzahlen
n_estimators_list = [10, 50, 100, 200]
best_accuracy = 0
best_model = None
for n_estimators in n_estimators_list:
model = RandomForestClassifier(n_estimators=n_estimators, random_state=42)
model.fit(X_train, y_train)
# Vorhersagen
y_test_pred = model.predict(X_test)
accuracy = accuracy_score(y_test, y_test_pred)
print(f"Trees: {n_estimators}, Test Accuracy: {accuracy:.3f}")
if accuracy > best_accuracy:
best_accuracy = accuracy
best_model = model
print(f"\nBest Random Forest Accuracy: {best_accuracy:.3f}")
# Feature Importance
feature_importance = best_model.feature_importances_
print("Top 5 Feature Importances:")
for i, importance in sorted(enumerate(feature_importance), key=lambda x: x[1], reverse=True)[:5]:
print(f" Feature {i}: {importance:.3f}")
self.models['random_forest'] = best_model
self.results['random_forest'] = {'test_accuracy': best_accuracy}
return X_train, X_test, y_train, y_test
# Model Comparison
def compare_models(self):
print("\n=== Model Comparison ===")
# Vergleichstabelle
comparison_data = []
for model_name, results in self.results.items():
if 'test_accuracy' in results:
comparison_data.append({
'Model': model_name,
'Test Accuracy': f"{results['test_accuracy']:.3f}"
})
elif 'test_mse' in results:
comparison_data.append({
'Model': model_name,
'Test MSE': f"{results['test_mse']:.2f}",
'R² Score': f"{results['r2_score']:.3f}"
})
df = pd.DataFrame(comparison_data)
print(df.to_string(index=False))
return df
# Demo ausführen
def supervised_learning_demo():
demo = SupervisedLearningDemo()
# Lineare Regression
X_lr_train, X_lr_test, y_lr_train, y_lr_test, y_lr_pred = demo.linear_regression_demo()
# Logistische Regression
X_log_train, X_log_test, y_log_train, y_log_test, y_log_pred = demo.logistic_regression_demo()
# Decision Tree
X_dt_train, X_dt_test, y_dt_train, y_dt_test = demo.decision_tree_demo()
# Random Forest
X_rf_train, X_rf_test, y_rf_train, y_rf_test = demo.random_forest_demo()
# Modelle vergleichen
comparison = demo.compare_models()
return demo, comparison
if __name__ == "__main__":
demo, comparison = supervised_learning_demo()2. Aprendizaje no supervisado con Python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans, DBSCAN, AgglomerativeClustering
from sklearn.decomposition import PCA, TSNE
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import silhouette_score
from sklearn.datasets import make_blobs, make_moons, load_iris
# Unsupervised Learning Demo
class UnsupervisedLearningDemo:
def __init__(self):
self.models = {}
self.results = {}
# K-Means Clustering
def kmeans_demo(self):
print("=== K-Means Clustering Demo ===")
# Synthetische Cluster-Daten
X, y_true = make_blobs(n_samples=300, centers=4, cluster_std=0.8, random_state=42)
# Features skalieren
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
# K-Means mit verschiedenen Cluster-Anzahlen
cluster_range = range(2, 8)
silhouette_scores = []
inertias = []
for k in cluster_range:
kmeans = KMeans(n_clusters=k, random_state=42, n_init=10)
cluster_labels = kmeans.fit_predict(X_scaled)
# Silhouette Score
silhouette_avg = silhouette_score(X_scaled, cluster_labels)
silhouette_scores.append(silhouette_avg)
# Inertia (Within-cluster sum of squares)
inertias.append(kmeans.inertia_)
print(f"K={k}: Silhouette Score={silhouette_avg:.3f}, Inertia={inertia_avg:.1f}")
# Optimalen K-Wert basierend auf Silhouette Score
optimal_k = cluster_range[np.argmax(silhouette_scores)]
print(f"\nOptimal K based on Silhouette: {optimal_k}")
# Finale K-Means mit optimalem K
final_kmeans = KMeans(n_clusters=optimal_k, random_state=42, n_init=10)
final_labels = final_kmeans.fit_predict(X_scaled)
# Ergebnisse speichern
self.models['kmeans'] = final_kmeans
self.results['kmeans'] = {
'optimal_k': optimal_k,
'silhouette_score': max(silhouette_scores),
'inertia': final_kmeans.inertia_
}
return X_scaled, final_labels, y_true
# DBSCAN Clustering
def dbscan_demo(self):
print("\n=== DBSCAN Clustering Demo ===")
# Nicht-kugelförmige Daten
X, y_true = make_moons(n_samples=200, noise=0.1, random_state=42)
# Features skalieren
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
# DBSCAN mit verschiedenen eps-Werten
eps_values = [0.2, 0.3, 0.4, 0.5]
min_samples = 5
for eps in eps_values:
dbscan = DBSCAN(eps=eps, min_samples=min_samples)
cluster_labels = dbscan.fit_predict(X_scaled)
# Anzahl der Cluster (ignoriert Rauschen)
n_clusters = len(set(cluster_labels)) - (1 if -1 in cluster_labels else 0)
n_noise = list(cluster_labels).count(-1)
if n_clusters > 1:
silhouette_avg = silhouette_score(X_scaled, cluster_labels)
else:
silhouette_avg = -1
print(f"eps={eps}: Clusters={n_clusters}, Noise={n_noise}, Silhouette={silhouette_avg:.3f}")
# Bestes DBSCAN
best_dbscan = DBSCAN(eps=0.3, min_samples=min_samples)
best_labels = best_dbscan.fit_predict(X_scaled)
self.models['dbscan'] = best_dbscan
self.results['dbscan'] = {
'n_clusters': len(set(best_labels)) - (1 if -1 in best_labels else 0),
'n_noise': list(best_labels).count(-1)
}
return X_scaled, best_labels, y_true
# Hierarchical Clustering
def hierarchical_clustering_demo(self):
print("\n=== Hierarchical Clustering Demo ===")
# Iris Dataset
iris = load_iris()
X = iris.data
y_true = iris.target
# Features skalieren
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
# Agglomerative Clustering mit verschiedenen Linkage-Methoden
linkage_methods = ['ward', 'complete', 'average', 'single']
for linkage in linkage_methods:
clustering = AgglomerativeClustering(n_clusters=3, linkage=linkage)
cluster_labels = clustering.fit_predict(X_scaled)
silhouette_avg = silhouette_score(X_scaled, cluster_labels)
print(f"Linkage={linkage}: Silhouette Score={silhouette_avg:.3f}")
# Bestes Linkage
best_clustering = AgglomerativeClustering(n_clusters=3, linkage='ward')
best_labels = best_clustering.fit_predict(X_scaled)
self.models['hierarchical'] = best_clustering
self.results['hierarchical'] = {
'silhouette_score': silhouette_score(X_scaled, best_labels)
}
return X_scaled, best_labels, y_true
# PCA (Principal Component Analysis)
def pca_demo(self):
print("\n=== PCA Demo ===")
# Hochdimensionale Daten
np.random.seed(42)
X = np.random.randn(100, 10)
# Korrelationen erzeugen
X[:, 1] = X[:, 0] * 0.8 + np.random.randn(100) * 0.2
X[:, 2] = X[:, 0] * 0.6 + np.random.randn(100) * 0.4
X[:, 3] = X[:, 1] * 0.7 + np.random.randn(100) * 0.3
# Features skalieren
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
# PCA mit verschiedenen Komponentenanzahlen
n_components_range = range(2, 11)
explained_variances = []
for n in n_components_range:
pca = PCA(n_components=n)
X_pca = pca.fit_transform(X_scaled)
total_explained_variance = np.sum(pca.explained_variance_ratio_)
explained_variances.append(total_explained_variance)
print(f"Components={n}: Explained Variance={total_explained_variance:.3f}")
# Optimal Anzahl basierend auf 95% Varianz
optimal_components = next(n for n, var in zip(n_components_range, explained_variances)
if var >= 0.95)
print(f"\nOptimal components for 95% variance: {optimal_components}")
# Finale PCA
final_pca = PCA(n_components=optimal_components)
X_pca_final = final_pca.fit_transform(X_scaled)
# Feature Contributions
print("Top contributing features for first component:")
feature_contributions = np.abs(final_pca.components_[0])
top_features = np.argsort(feature_contributions)[-3:][::-1]
for i, feature_idx in enumerate(top_features):
print(f" Feature {feature_idx}: {feature_contributions[feature_idx]:.3f}")
self.models['pca'] = final_pca
self.results['pca'] = {
'optimal_components': optimal_components,
'explained_variance': np.sum(final_pca.explained_variance_ratio_)
}
return X_scaled, X_pca_final
# t-SNE für Visualisierung
def tsne_demo(self):
print("\n=== t-SNE Demo ===")
# Iris Dataset für Visualisierung
iris = load_iris()
X = iris.data
y = iris.target
# Features skalieren
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
# t-SNE mit verschiedenen Perplexity-Werten
perplexity_values = [5, 15, 30, 50]
for perplexity in perplexity_values:
tsne = TSNE(n_components=2, perplexity=perplexity, random_state=42)
X_tsne = tsne.fit_transform(X_scaled)
print(f"Perplexity={perplexity}: K-Loss={tsne.kl_divergence_:.3f}")
# Bestes t-SNE
best_tsne = TSNE(n_components=2, perplexity=30, random_state=42)
X_tsne_final = best_tsne.fit_transform(X_scaled)
self.models['tsne'] = best_tsne
return X_scaled, X_tsne_final, y
# Clustering Evaluation
def evaluate_clustering(self, X, labels, true_labels=None):
print("\n=== Clustering Evaluation ===")
# Silhouette Score
if len(set(labels)) > 1:
silhouette_avg = silhouette_score(X, labels)
print(f"Silhouette Score: {silhouette_avg:.3f}")
else:
print("Silhouette Score: N/A (only one cluster)")
# Cluster-Statistiken
n_clusters = len(set(labels)) - (1 if -1 in labels else 0)
n_noise = list(labels).count(-1)
print(f"Number of clusters: {n_clusters}")
print(f"Number of noise points: {n_noise}")
# Cluster-Größen
if n_clusters > 0:
cluster_sizes = [np.sum(labels == i) for i in range(n_clusters)]
print(f"Cluster sizes: {cluster_sizes}")
print(f"Average cluster size: {np.mean(cluster_sizes):.1f}")
return {
'silhouette_score': silhouette_avg if len(set(labels)) > 1 else None,
'n_clusters': n_clusters,
'n_noise': n_noise
}
# Model Comparison
def compare_clustering_models(self):
print("\n=== Clustering Models Comparison ===")
comparison_data = []
for model_name, results in self.results.items():
if 'silhouette_score' in results:
comparison_data.append({
'Model': model_name,
'Silhouette Score': f"{results['silhouette_score']:.3f}"
})
elif 'optimal_k' in results:
comparison_data.append({
'Model': model_name,
'Optimal K': results['optimal_k'],
'Silhouette Score': f"{results['silhouette_score']:.3f}"
})
df = pd.DataFrame(comparison_data)
print(df.to_string(index=False))
return df
# Demo ausführen
def unsupervised_learning_demo():
demo = UnsupervisedLearningDemo()
# K-Means
X_km, labels_km, true_km = demo.kmeans_demo()
demo.evaluate_clustering(X_km, labels_km, true_km)
# DBSCAN
X_db, labels_db, true_db = demo.dbscan_demo()
demo.evaluate_clustering(X_db, labels_db, true_db)
# Hierarchical Clustering
X_hc, labels_hc, true_hc = demo.hierarchical_clustering_demo()
demo.evaluate_clustering(X_hc, labels_hc, true_hc)
# PCA
X_pca, X_pca_transformed = demo.pca_demo()
# t-SNE
X_tsne, X_tsne_transformed, y_tsne = demo.tsne_demo()
# Modelle vergleichen
comparison = demo.compare_clustering_models()
return demo, comparison
if __name__ == "__main__":
demo, comparison = unsupervised_learning_demo()3. Aprendizaje por refuerzo con Python
import numpy as np
import random
import matplotlib.pyplot as plt
from collections import defaultdict
# Reinforcement Learning Demo
class ReinforcementLearningDemo:
def __init__(self):
self.environments = {}
self.agents = {}
self.results = {}
# Grid World Environment
class GridWorld:
def __init__(self, width=4, height=4):
self.width = width
self.height = height
self.state = (0, 0) # Posición inicial
self.goal = (width-1, height-1) # Posición objetivo
self.obstacles = [(1, 1), (2, 2)] # Obstáculos
self.terminal_states = [self.goal]
def reset(self):
self.state = (0, 0)
return self.state
def step(self, action):
x, y = self.state
# Ejecutar acción
if action == 0: # Arriba
new_state = (x, max(0, y - 1))
elif action == 1: # Abajo
new_state = (x, min(self.height - 1, y + 1))
elif action == 2: # Izquierda
new_state = (max(0, x - 1), y)
elif action == 3: # Derecha
new_state = (min(self.width - 1, x + 1), y)
else:
new_state = self.state
# Verificar obstáculos
if new_state in self.obstacles:
new_state = self.state
# Calcular recompensa
if new_state == self.goal:
reward = 10
done = True
else:
reward = -1 # Penalización pequeña por cada paso
done = False
self.state = new_state
return new_state, reward, done
def get_valid_actions(self):
return [0, 1, 2, 3] # Arriba, Abajo, Izquierda, Derecha
def render(self):
grid = np.zeros((self.height, self.width))
# Marcar obstáculos
for obs in self.obstacles:
grid[obs[1], obs[0]] = -1
# Marcar objetivo
grid[self.goal[1], self.goal[0]] = 10
# Marcar posición actual
grid[self.state[1], self.state[0]] = 1
print("Grid World:")
print(grid)
print(f"Position: {self.state}, Goal: {self.goal}")
# Q-Learning Agent
class QLearningAgent:
def __init__(self, state_space_size, action_space_size, learning_rate=0.1,
discount_factor=0.9, epsilon=0.1):
self.state_space_size = state_space_size
self.action_space_size = action_space_size
self.learning_rate = learning_rate
self.discount_factor = discount_factor
self.epsilon = epsilon
# Inicializar tabla Q
self.q_table = defaultdict(lambda: np.zeros(action_space_size))
def get_state_index(self, state):
# Convertir coordenadas 2D a índice 1D
x, y = state
return y * 4 + x
def choose_action(self, state, valid_actions):
state_idx = self.get_state_index(state)
# Estrategia epsilon-greedy
if random.random() < self.epsilon:
return random.choice(valid_actions)
else:
q_values = self.q_table[state_idx]
valid_q_values = [q_values[action] for action in valid_actions]
max_q = max(valid_q_values)
# Si hay Q-valores iguales, elegir aleatoriamente
best_actions = [action for action in valid_actions
if q_values[action] == max_q]
return random.choice(best_actions)
def update_q_value(self, state, action, reward, next_state, valid_next_actions):
state_idx = self.get_state_index(state)
next_state_idx = self.get_state_index(next_state)
# Actualizar Q-valor
current_q = self.q_table[state_idx][action]
if len(valid_next_actions) > 0:
max_next_q = max([self.q_table[next_state_idx][a] for a in valid_next_actions])
else:
max_next_q = 0
new_q = current_q + self.learning_rate * (
reward + self.discount_factor * max_next_q - current_q
)
self.q_table[state_idx][action] = new_q
def get_policy(self):
policy = {}
for state_idx in self.q_table.keys():
y = state_idx // 4
x = state_idx % 4
state = (x, y)
valid_actions = [0, 1, 2, 3] # Todas las acciones son válidas
q_values = self.q_table[state_idx]
best_action = np.argmax(q_values)
policy[state] = best_action
return policy
# Demo Q-Learning
def q_learning_demo(self):
print("=== Q-Learning Demo ===")
# Crear environment y agent
env = self.GridWorld(width=4, height=4)
agent = self.QLearningAgent(state_space_size=16, action_space_size=4)
# Parámetros de entrenamiento
episodes = 1000
max_steps_per_episode = 100
# Entrenamiento
episode_rewards = []
for episode in range(episodes):
state = env.reset()
total_reward = 0
done = False
steps = 0
while not done and steps < max_steps_per_episode:
valid_actions = env.get_valid_actions()
action = agent.choose_action(state, valid_actions)
next_state, reward, done = env.step(action)
valid_next_actions = env.get_valid_actions()
# Actualizar Q-valor
agent.update_q_value(state, action, reward, next_state, valid_next_actions)
state = next_state
total_reward += reward
steps += 1
episode_rewards.append(total_reward)
if episode % 100 == 0:
avg_reward = np.mean(episode_rewards[-100:])
print(f"Episode {episode}: Average Reward (last 100): {avg_reward:.2f}")
# Analizar resultados
final_policy = agent.get_policy()
print(f"\nFinal Policy:")
for state, action in final_policy.items():
action_names = {0: 'Arriba', 1: 'Abajo', 2: 'Izquierda', 3: 'Derecha'}
print(f"State {state}: {action_names[action]}")
# Mostrar tabla Q
print(f"\nQ-Table (selected states):")
for state_idx in [0, 5, 10, 15]: # Esquinas
y = state_idx // 4
x = state_idx % 4
state = (x, y)
q_values = agent.q_table[state_idx]
print(f"State {state}: {q_values}")
self.environments['gridworld'] = env
self.agents['qlearning'] = agent
self.results['qlearning'] = {
'episodes': episodes,
'final_avg_reward': np.mean(episode_rewards[-100:]),
'q_table_size': len(agent.q_table)
}
return episode_rewards
# Environment CartPole simple
class CartPoleSimple:
def __init__(self):
self.angle = 0 # Ángulo del poste
self.angular_velocity = 0 # Velocidad angular
self.gravity = 9.8
self.pole_length = 1.0
self.dt = 0.1
def reset(self):
self.angle = random.uniform(-0.1, 0.1)
self.angular_velocity = 0
return self.get_state()
def get_state(self):
return (self.angle, self.angular_velocity)
def step(self, action):
# Acciones: 0 = Izquierda, 1 = Derecha
force = -10 if action == 0 else 10
# Actualización de física (simplificada)
angular_acceleration = (self.gravity / self.pole_length) * np.sin(self.angle) + force
self.angular_velocity += angular_acceleration * self.dt
self.angle += self.angular_velocity * self.dt
# Recompensa y condición de finalización
if abs(self.angle) > np.pi / 4: # El poste se cae
reward = -10
done = True
else:
reward = 1 # Recompensa por mantener equilibrio
done = False
return self.get_state(), reward, done
def render(self):
print(f"Angle: {self.angle:.3f} rad ({np.degrees(self.angle):.1f}°), "
f"Angular Velocity: {self.angular_velocity:.3f}")
# Policy Gradient Agent (simplificado)
class PolicyGradientAgent:
def __init__(self, state_dim=2, action_dim=2, learning_rate=0.01):
self.state_dim = state_dim
self.action_dim = action_dim
self.learning_rate = learning_rate
# Política lineal simple
self.weights = np.random.randn(state_dim, action_dim) * 0.1
def get_action_probabilities(self, state):
# Softmax sobre combinación lineal
logits = np.dot(state, self.weights)
exp_logits = np.exp(logits - np.max(logits))
return exp_logits / np.sum(exp_logits)
def choose_action(self, state):
action_probs = self.get_action_probabilities(state)
return np.random.choice(self.action_dim, p=action_probs)
def update_policy(self, states, actions, rewards):
# Actualización simplificada de Policy Gradient
for state, action, reward in zip(states, actions, rewards):
action_probs = self.get_action_probabilities(state)
# Calcular gradiente
grad = np.zeros_like(self.weights)
for a in range(self.action_dim):
if a == action:
grad[:, a] = state * (1 - action_probs[a])
else:
grad[:, a] = -state * action_probs[a]
# Actualizar
self.weights += self.learning_rate * reward * grad
# Demo Policy Gradient
def policy_gradient_demo(self):
print("\n=== Policy Gradient Demo ===")
env = self.CartPoleSimple()
agent = self.PolicyGradientAgent()
episodes = 500
episode_rewards = []
for episode in range(episodes):
state = env.reset()
states, actions, rewards = [], [], []
total_reward = 0
done = False
steps = 0
max_steps = 100
while not done and steps < max_steps:
action = agent.choose_action(state)
next_state, reward, done = env.step(action)
states.append(state)
actions.append(action)
rewards.append(reward)
state = next_state
total_reward += reward
steps += 1
# Actualizar política
agent.update_policy(states, actions, rewards)
episode_rewards.append(total_reward)
if episode % 50 == 0:
avg_reward = np.mean(episode_rewards[-50:])
print(f"Episode {episode}: Average Reward (last 50): {avg_reward:.2f}")
# Evaluación final
print(f"\nFinal Evaluation:")
state = env.reset()
for step in range(20):
action_probs = agent.get_action_probabilities(state)
action = np.argmax(action_probs)
state, reward, done = env.step(action)
env.render()
if done:
print("Episode finished!")
break
self.environments['cartpole'] = env
self.agents['policy_gradient'] = agent
self.results['policy_gradient'] = {
'episodes': episodes,
'final_avg_reward': np.mean(episode_rewards[-50:])
}
return episode_rewards
# Comparación de modelos
def compare_rl_models(self):
print("\n=== Reinforcement Learning Models Comparison ===")
comparison_data = []
for model_name, results in self.results.items():
comparison_data.append({
'Model': model_name,
'Episodes': results['episodes'],
'Final Avg Reward': f"{results['final_avg_reward']:.2f}"
})
df = pd.DataFrame(comparison_data)
print(df.to_string(index=False))
return df
# Ejecutar demo
def reinforcement_learning_demo():
demo = ReinforcementLearningDemo()
# Q-Learning
q_rewards = demo.q_learning_demo()
# Policy Gradient
pg_rewards = demo.policy_gradient_demo()
# Comparar modelos
comparison = demo.compare_rl_models()
# Visualizar recompensas
plt.figure(figsize=(12, 4))
plt.subplot(1, 2, 1)
plt.plot(q_rewards)
plt.title('Q-Learning Rewards')
plt.xlabel('Episode')
plt.ylabel('Total Reward')
plt.subplot(1, 2, 2)
plt.plot(pg_rewards)
plt.title('Policy Gradient Rewards')
plt.xlabel('Episode')
plt.ylabel('Total Reward')
plt.tight_layout()
plt.show()
return demo, comparison
if __name__ == "__main__":
demo, comparison = reinforcement_learning_demo()
Resumen de tipos de Machine Learning
| Tipo | Datos | Objetivo | Ejemplos | Algoritmos |
|---|---|---|---|---|
| Supervised | Etiquetados | Predicción | Clasificación, Regresión | Linear Regression, Decision Trees |
| Unsupervised | Sin etiquetar | Encontrar patrones | Clustering, Dimensionality Reduction | K-Means, PCA |
| Reinforcement | Entorno | Maximizar recompensa | Juegos, Robótica | Q-Learning, Policy Gradients |
Comparación de Algoritmos
Supervised Learning
| Algoritmo | Tipo | Complejidad | Ventajas | Desventajas |
|---|---|---|---|---|
| Linear Regression | Regresión | O(n) | Interpretable | Solo relaciones lineales |
| Logistic Regression | Clasificación | O(n) | Rápida, interpretable | Linealidad |
| Decision Trees | Ambos | O(n log n) | Interpretable | Sobreajuste |
| Random Forest | Ambos | O(n log n) | Robusto, preciso | Complejo |
| SVM | Ambos | O(n²) | Alta precisión | Escalabilidad limitada |
Unsupervised Learning
| Algoritmo | Tipo | Complejidad | Ventajas | Desventajas |
|---|---|---|---|---|
| K-Means | Clustering | O(n k i) | Rápido | Solo clusters esféricos |
| DBSCAN | Clustering | O(n log n) | Formas arbitrarias | Sensible a parámetros |
| PCA | Dimensionality | O(n d²) | Reduce dimensiones | Linealidad |
| t-SNE | Visualization | O(n²) | No lineal | Lento |
Reinforcement Learning
| Algoritmo | Tipo | Complejidad | Ventajas | Desventajas |
|---|---|---|---|---|
| Q-Learning | Model-free | O(s a) | Simple | Espacios discretos |
| Deep Q-Network | Model-free | O(n) | Continuo | Inestable |
| Policy Gradients | Model-free | O(n) | Estocástico | Alta varianza |
Flujo de trabajo ML
1. Recopilación de datos
# Identificar fuentes de datos
# Asegurar calidad
# Considerar ética y privacidad2. Preparación de datos
# Cleaning: Tratar valores faltantes
# Feature Engineering: Crear nuevas características
# Scaling: Normalización/Estandarización
# Splitting: Train/Validación/Test3. Selección de modelo
# Identificar tipo de problema
# Crear modelo base
# Probar múltiples algoritmos
# Optimizar hiperparámetros4. Entrenamiento
# Usar Cross-Validation
# Evitar sobreajuste
# Implementar Early Stopping
# Monitorear métricas5. Evaluación
# Medir performance
# Analizar errores
# Probar robustez
# Evaluar valor de negocioMétricas de evaluación
Clasificación
- Accuracy: Predicciones correctas / Total
- Precision: True Positives / (TP + FP)
- Recall: True Positives / (TP + FN)
- F1-Score: Media armónica de Precision y Recall
- ROC-AUC: Area Under ROC Curve
Regresión
- MSE: Mean Squared Error
- RMSE: Root Mean Squared Error
- MAE: Mean Absolute Error
- R²: Coeficiente de determinación
Clustering
- Silhouette Score: Calidad del cluster
- Davies-Bouldin Index: Separación entre clusters
- Calinski-Harabasz: Proporción entre clusters
Sobreajuste vs Subajuste
Sobreajuste
- Síntomas: Alta precisión en entrenamiento, baja en test
- Causas: Modelo demasiado complejo, pocos datos
- Soluciones: Regularización, más datos, modelo más simple
Subajuste
- Síntomas: Baja precisión en ambos conjuntos
- Causas: Modelo demasiado simple, pocas características
- Soluciones: Modelo más complejo, Feature Engineering
Feature Engineering
Técnicas
# Polynomial Features
# Interaction Terms
# Binning/Discretization
# Log-Transformation
# One-Hot Encoding
# Target Encoding
# Feature SelectionAutomatización
# AutoML Tools
# Feature Importance Analysis
# Recursive Feature Elimination
# Genetic AlgorithmsVentajas y Desventajas
Ventajas de Machine Learning
- Automatización: Reducir trabajo manual
- Reconocimiento de patrones: Encontrar relaciones complejas
- Escalabilidad: Procesar grandes volúmenes de datos
- Adaptabilidad: Ajustarse a nuevos datos
Desventajas
- Dependencia de datos: La calidad depende de los datos
- Complejidad: Problema de la caja negra
- Costos computacionales: El entrenamiento puede ser costoso
- Ética: Considerar sesgo y equidad
Preguntas frecuentes de examen
-
¿Cuál es la diferencia entre Supervised y Unsupervised Learning? Supervised Learning usa datos etiquetados para predicciones, Unsupervised Learning encuentra patrones en datos sin etiquetar.
-
¡Explica el sobreajuste y cómo evitarlo! El sobreajuste es adaptarse demasiado a los datos de entrenamiento. Se evita con regularización, más datos y Cross-Validation.
-
¿Cuándo se usa Reinforcement Learning? Cuando un agente debe aprender interactuando con un entorno para maximizar su recompensa.
-
¿Cuál es la diferencia entre Clasificación y Regresión? Clasificación predice clases discretas, Regresión predice valores continuos.