465 lines
14 KiB
Markdown
465 lines
14 KiB
Markdown
# Guía para Añadir Vinted y Buyee
|
|
|
|
Esta guía específica te ayudará a implementar las plataformas Vinted y Buyee en el monitor.
|
|
|
|
## Vinted
|
|
|
|
### 1. Investigar la API de Vinted
|
|
|
|
Vinted no tiene una API pública oficial, pero tiene una API interna que puedes usar:
|
|
|
|
**URL Base**: `https://www.vinted.es/api/v2/catalog/items`
|
|
|
|
**Parámetros comunes**:
|
|
- `search_text`: Términos de búsqueda
|
|
- `catalog_ids`: IDs de categorías
|
|
- `price_from`: Precio mínimo
|
|
- `price_to`: Precio máximo
|
|
- `currency`: Moneda (EUR, USD, etc.)
|
|
- `order`: Ordenar por (newest_first, price_low_to_high, etc.)
|
|
- `per_page`: Artículos por página
|
|
|
|
### 2. Ejemplo de Implementación
|
|
|
|
```python
|
|
# platforms/vinted_platform.py
|
|
import requests
|
|
import logging
|
|
import time
|
|
from datetime import datetime
|
|
from platforms.base_platform import BasePlatform
|
|
from models.article import Article
|
|
|
|
REQUEST_RETRY_TIME = 5
|
|
|
|
class VintedPlatform(BasePlatform):
|
|
"""Vinted marketplace platform implementation"""
|
|
|
|
def __init__(self, item_monitor):
|
|
super().__init__(item_monitor)
|
|
self.logger = logging.getLogger(__name__)
|
|
|
|
def get_platform_name(self):
|
|
return "vinted"
|
|
|
|
def create_url(self):
|
|
"""Construir URL de búsqueda de Vinted"""
|
|
url = "https://www.vinted.es/api/v2/catalog/items"
|
|
params = []
|
|
|
|
# Query de búsqueda
|
|
search_query = self._item_monitor.get_search_query()
|
|
params.append(f"search_text={search_query}")
|
|
|
|
# Ordenar por más reciente
|
|
params.append("order=newest_first")
|
|
|
|
# Precio
|
|
if self._item_monitor.get_min_price() != 0:
|
|
params.append(f"price_from={self._item_monitor.get_min_price()}")
|
|
|
|
if self._item_monitor.get_max_price() != 0:
|
|
params.append(f"price_to={self._item_monitor.get_max_price()}")
|
|
|
|
# Moneda
|
|
params.append("currency=EUR")
|
|
|
|
# Resultados por página
|
|
params.append("per_page=50")
|
|
|
|
return url + "?" + "&".join(params)
|
|
|
|
def get_request_headers(self):
|
|
"""Headers para Vinted"""
|
|
headers = super().get_request_headers()
|
|
headers['Accept'] = 'application/json'
|
|
headers['Accept-Language'] = 'es-ES'
|
|
return headers
|
|
|
|
def fetch_articles(self):
|
|
"""Obtener artículos desde Vinted"""
|
|
url = self.create_url()
|
|
|
|
while True:
|
|
try:
|
|
headers = self.get_request_headers()
|
|
response = requests.get(url, headers=headers)
|
|
response.raise_for_status()
|
|
break
|
|
except requests.exceptions.RequestException as err:
|
|
self.logger.error(f"Vinted Request Exception: {err}")
|
|
time.sleep(REQUEST_RETRY_TIME)
|
|
|
|
json_response = response.json()
|
|
json_items = json_response.get('items', [])
|
|
articles = self.parse_response(json_items)
|
|
return articles
|
|
|
|
def parse_response(self, json_items):
|
|
"""Parsear respuesta de Vinted"""
|
|
articles = []
|
|
for json_article in json_items:
|
|
article = self._parse_single_article(json_article)
|
|
if article:
|
|
articles.append(article)
|
|
return articles
|
|
|
|
def _parse_single_article(self, json_data):
|
|
"""Parsear un artículo individual de Vinted"""
|
|
try:
|
|
# Extraer imágenes
|
|
images = []
|
|
if 'photo' in json_data and json_data['photo']:
|
|
images.append(json_data['photo'].get('url', ''))
|
|
|
|
# Más imágenes si están disponibles
|
|
if 'photos' in json_data:
|
|
for photo in json_data['photos'][:3]:
|
|
if photo.get('url'):
|
|
images.append(photo['url'])
|
|
|
|
# Convertir fecha
|
|
updated_at = json_data.get('updated_at', '')
|
|
try:
|
|
dt = datetime.fromisoformat(updated_at.replace('Z', '+00:00'))
|
|
modified_at = dt.strftime("%Y-%m-%d %H:%M:%S")
|
|
except:
|
|
modified_at = updated_at
|
|
|
|
# Precio
|
|
price = float(json_data.get('price', {}).get('amount', 0))
|
|
currency = json_data.get('price', {}).get('currency_code', 'EUR')
|
|
|
|
# Ubicación
|
|
location = json_data.get('user', {}).get('city', 'Unknown')
|
|
|
|
# URL
|
|
article_url = f"https://www.vinted.es/items/{json_data.get('id')}"
|
|
|
|
# Envíos (Vinted suele permitir envíos)
|
|
allows_shipping = True
|
|
|
|
return Article(
|
|
id=str(json_data['id']),
|
|
title=json_data.get('title', ''),
|
|
description=json_data.get('description', ''),
|
|
price=price,
|
|
currency=currency,
|
|
location=location,
|
|
allows_shipping=allows_shipping,
|
|
url=article_url,
|
|
images=images[:3], # Máximo 3 imágenes
|
|
modified_at=modified_at,
|
|
platform=self.get_platform_name()
|
|
)
|
|
except (KeyError, ValueError) as e:
|
|
self.logger.error(f"Error parsing Vinted article: {e}")
|
|
return None
|
|
```
|
|
|
|
### 3. Registrar Vinted
|
|
|
|
En `platforms/platform_factory.py`:
|
|
```python
|
|
from platforms.vinted_platform import VintedPlatform
|
|
|
|
class PlatformFactory:
|
|
_platforms = {
|
|
'wallapop': WallapopPlatform,
|
|
'vinted': VintedPlatform,
|
|
}
|
|
```
|
|
|
|
### 4. Usar en workers.json
|
|
|
|
```json
|
|
{
|
|
"name": "Gameboy en Vinted",
|
|
"platform": "vinted",
|
|
"search_query": "gameboy",
|
|
"min_price": 10,
|
|
"max_price": 100,
|
|
"thread_id": 10
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Buyee (Yahoo Auctions Japan)
|
|
|
|
### 1. Investigar la API de Buyee
|
|
|
|
Buyee es un servicio proxy para Yahoo Auctions Japan. Opciones:
|
|
|
|
**Opción A - Yahoo Auctions API** (si tienes acceso):
|
|
- URL: `https://auctions.yahooapis.jp/AuctionWebService/V2/search`
|
|
- Requiere API key
|
|
|
|
**Opción B - Web Scraping** (más común):
|
|
- URL: `https://buyee.jp/yahoo/auction/search/query/SEARCH_QUERY`
|
|
- Parsear HTML con BeautifulSoup
|
|
|
|
### 2. Ejemplo de Implementación (Web Scraping)
|
|
|
|
```python
|
|
# platforms/buyee_platform.py
|
|
import requests
|
|
import logging
|
|
import time
|
|
from datetime import datetime
|
|
from bs4 import BeautifulSoup
|
|
from platforms.base_platform import BasePlatform
|
|
from models.article import Article
|
|
|
|
REQUEST_RETRY_TIME = 5
|
|
|
|
class BuyeePlatform(BasePlatform):
|
|
"""Buyee (Yahoo Auctions Japan) marketplace platform implementation"""
|
|
|
|
def __init__(self, item_monitor):
|
|
super().__init__(item_monitor)
|
|
self.logger = logging.getLogger(__name__)
|
|
|
|
def get_platform_name(self):
|
|
return "buyee"
|
|
|
|
def create_url(self):
|
|
"""Construir URL de búsqueda de Buyee"""
|
|
search_query = self._item_monitor.get_search_query()
|
|
|
|
# URL base de Buyee
|
|
url = f"https://buyee.jp/yahoo/auction/search/query/{search_query}"
|
|
|
|
params = []
|
|
|
|
# Precio
|
|
if self._item_monitor.get_min_price() != 0:
|
|
params.append(f"min_price={self._item_monitor.get_min_price()}")
|
|
|
|
if self._item_monitor.get_max_price() != 0:
|
|
params.append(f"max_price={self._item_monitor.get_max_price()}")
|
|
|
|
# Ordenar por más reciente
|
|
params.append("sort=end_time&order=a")
|
|
|
|
if params:
|
|
url += "?" + "&".join(params)
|
|
|
|
return url
|
|
|
|
def get_request_headers(self):
|
|
"""Headers para Buyee"""
|
|
headers = super().get_request_headers()
|
|
headers['Accept'] = 'text/html,application/xhtml+xml'
|
|
headers['Accept-Language'] = 'en-US,en;q=0.9,ja;q=0.8'
|
|
return headers
|
|
|
|
def fetch_articles(self):
|
|
"""Obtener artículos desde Buyee"""
|
|
url = self.create_url()
|
|
|
|
while True:
|
|
try:
|
|
headers = self.get_request_headers()
|
|
response = requests.get(url, headers=headers)
|
|
response.raise_for_status()
|
|
break
|
|
except requests.exceptions.RequestException as err:
|
|
self.logger.error(f"Buyee Request Exception: {err}")
|
|
time.sleep(REQUEST_RETRY_TIME)
|
|
|
|
# Parsear HTML
|
|
soup = BeautifulSoup(response.text, 'html.parser')
|
|
articles = self.parse_response(soup)
|
|
return articles
|
|
|
|
def parse_response(self, soup):
|
|
"""Parsear HTML de Buyee"""
|
|
articles = []
|
|
|
|
# Los selectores pueden cambiar, esto es un ejemplo
|
|
# Necesitarás investigar la estructura HTML actual
|
|
items = soup.select('.product-item') or soup.select('.item')
|
|
|
|
for item in items:
|
|
article = self._parse_single_article(item)
|
|
if article:
|
|
articles.append(article)
|
|
|
|
return articles
|
|
|
|
def _parse_single_article(self, item_element):
|
|
"""Parsear un artículo individual de Buyee"""
|
|
try:
|
|
# NOTA: Estos selectores son ejemplos, necesitas verificar la estructura real
|
|
title_elem = item_element.select_one('.product-title') or item_element.select_one('.title')
|
|
title = title_elem.text.strip() if title_elem else "No title"
|
|
|
|
price_elem = item_element.select_one('.product-price') or item_element.select_one('.price')
|
|
price_text = price_elem.text.strip() if price_elem else "0"
|
|
# Limpiar precio: "¥1,500" -> 1500
|
|
price = float(price_text.replace('¥', '').replace(',', '').strip())
|
|
|
|
link_elem = item_element.select_one('a[href]')
|
|
url = link_elem['href'] if link_elem else ""
|
|
if url and not url.startswith('http'):
|
|
url = f"https://buyee.jp{url}"
|
|
|
|
# Extraer ID de la URL
|
|
item_id = url.split('/')[-1] if url else "unknown"
|
|
|
|
# Imagen
|
|
img_elem = item_element.select_one('img')
|
|
images = [img_elem['src']] if img_elem and 'src' in img_elem.attrs else []
|
|
|
|
# Descripción (si está disponible)
|
|
desc_elem = item_element.select_one('.description') or item_element.select_one('.desc')
|
|
description = desc_elem.text.strip() if desc_elem else ""
|
|
|
|
return Article(
|
|
id=item_id,
|
|
title=title,
|
|
description=description,
|
|
price=price,
|
|
currency="JPY",
|
|
location="Japan",
|
|
allows_shipping=True, # Buyee siempre permite envíos internacionales
|
|
url=url,
|
|
images=images,
|
|
modified_at=datetime.now().strftime("%Y-%m-%d %H:%M:%S"),
|
|
platform=self.get_platform_name()
|
|
)
|
|
except Exception as e:
|
|
self.logger.error(f"Error parsing Buyee article: {e}")
|
|
return None
|
|
```
|
|
|
|
### 3. Instalar BeautifulSoup
|
|
|
|
```bash
|
|
pip install beautifulsoup4
|
|
```
|
|
|
|
Añadir a `requirements.txt`:
|
|
```
|
|
beautifulsoup4>=4.12.0
|
|
```
|
|
|
|
### 4. Registrar Buyee
|
|
|
|
En `platforms/platform_factory.py`:
|
|
```python
|
|
from platforms.buyee_platform import BuyeePlatform
|
|
|
|
class PlatformFactory:
|
|
_platforms = {
|
|
'wallapop': WallapopPlatform,
|
|
'vinted': VintedPlatform,
|
|
'buyee': BuyeePlatform,
|
|
}
|
|
```
|
|
|
|
### 5. Usar en workers.json
|
|
|
|
```json
|
|
{
|
|
"name": "Retro Games en Buyee",
|
|
"platform": "buyee",
|
|
"search_query": "ファミコン",
|
|
"min_price": 1000,
|
|
"max_price": 50000,
|
|
"thread_id": 11
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Consideraciones Importantes
|
|
|
|
### Rate Limiting
|
|
- Vinted y Buyee pueden tener límites de peticiones
|
|
- Considera añadir delays entre peticiones
|
|
- Usa proxies si es necesario
|
|
|
|
### Web Scraping vs API
|
|
- **Vinted**: Tiene API interna, relativamente estable
|
|
- **Buyee**: Web scraping puede romperse con cambios en el sitio
|
|
- Considera usar Yahoo Auctions API oficial si tienes acceso
|
|
|
|
### Monedas
|
|
- Vinted: EUR (España), USD (USA), GBP (UK), etc.
|
|
- Buyee: JPY (Yenes japoneses)
|
|
- El sistema ya soporta diferentes monedas en el modelo Article
|
|
|
|
### Ubicaciones Geográficas
|
|
- Vinted: Soporta filtrado por ubicación (como Wallapop)
|
|
- Buyee: Todos los artículos son de Japón
|
|
|
|
### Condición de Artículos
|
|
- Vinted: Tiene estados similares a Wallapop
|
|
- Buyee: Depende del vendedor en Yahoo Auctions
|
|
|
|
---
|
|
|
|
## Testing
|
|
|
|
Para probar tus implementaciones:
|
|
|
|
```python
|
|
from platforms.platform_factory import PlatformFactory
|
|
from datalayer.item_monitor import ItemMonitor
|
|
|
|
# Test Vinted
|
|
vinted_config = {
|
|
"name": "Test Vinted",
|
|
"platform": "vinted",
|
|
"search_query": "gameboy"
|
|
}
|
|
item = ItemMonitor.load_from_json(vinted_config)
|
|
vinted = PlatformFactory.create_platform("vinted", item)
|
|
articles = vinted.fetch_articles()
|
|
print(f"Found {len(articles)} articles on Vinted")
|
|
|
|
# Test Buyee
|
|
buyee_config = {
|
|
"name": "Test Buyee",
|
|
"platform": "buyee",
|
|
"search_query": "ゲームボーイ"
|
|
}
|
|
item = ItemMonitor.load_from_json(buyee_config)
|
|
buyee = PlatformFactory.create_platform("buyee", item)
|
|
articles = buyee.fetch_articles()
|
|
print(f"Found {len(articles)} articles on Buyee")
|
|
```
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Vinted no devuelve resultados
|
|
- Verifica que la URL de la API no haya cambiado
|
|
- Comprueba los headers (User-Agent, Accept-Language)
|
|
- Prueba la URL directamente en el navegador
|
|
|
|
### Buyee parseo falla
|
|
- La estructura HTML puede cambiar
|
|
- Inspecciona la página web y actualiza los selectores CSS
|
|
- Considera usar Yahoo Auctions API oficial
|
|
|
|
### Errores 403/429
|
|
- Añade más delays entre peticiones
|
|
- Usa User-Agent realista
|
|
- Considera rotar IPs o usar proxies
|
|
|
|
---
|
|
|
|
## Próximos Pasos
|
|
|
|
1. Implementa Vinted primero (más fácil, tiene API)
|
|
2. Prueba con búsquedas reales
|
|
3. Implementa Buyee (más complejo, scraping)
|
|
4. Ajusta los selectores según la estructura actual
|
|
5. Añade manejo de errores específico
|
|
6. Documenta cualquier peculiaridad de la plataforma
|
|
|