3.1. xboinc
3.1.1. Submodules
3.1.2. Attributes
3.1.3. Classes
Class to retrieve and manage results from Xboinc simulations. |
|
A class to manage jobs for submission to the Xboinc server. |
3.1.4. Functions
|
Check if a user is subscribed to the work unit database. |
|
List all work units for the registered users with an optional status filter. |
|
Generate the Xboinc executable. |
|
Generate all source files needed to compile the Xboinc executable. |
|
Remove a user from the BOINC server and dev server. This is |
|
Register a user to the BOINC server and dev server, by declaring |
3.1.5. Package Contents
- check_user_subscription(user: str) bool[source]
Check if a user is subscribed to the work unit database.
- query_registered_work_units(status: str | None = None, dev_server: bool = False) pandas.DataFrame[source]
List all work units for the registered users with an optional status filter.
- generate_executable(*, keep_source=False, clean=True, vcpkg_root=None, target_triplet='x64-linux')[source]
Generate the Xboinc executable.
- Parameters:
keep_source (bool, optional) – Whether or not to keep the source files. Defaults to False.
clean (bool, optional) – Whether or not to clean the make directory. Defaults to True.
vcpkg_root (pathlib.Path, optional) – The path to the local VCPKG installation. If none, an executable without the BOINC API is generated. Defaults to None.
target_triplet (string, optional) – The target architecture to compile to. Currently x64-linux and x64-mingw-static (i.e. Windows 64-bit) are supported. Defaults to x64-linux.
- Return type:
None
- generate_executable_source(*, overwrite=False, _context=None)[source]
Generate all source files needed to compile the Xboinc executable.
- Parameters:
overwrite (bool, optional) – Whether or not to overwrite existing source files.
- Return type:
None
- deregister(user)[source]
Remove a user from the BOINC server and dev server. This is not instantaneous, as the BOINC server periodically parses the users to remove.
- Parameters:
user (string) – Name of the user to deregister.
- Return type:
None.
- register(user, directory)[source]
Register a user to the BOINC server and dev server, by declaring the username and user boinc directory, and giving access rights to the BOINC admin process. This is not instantaneous, as the BOINC server periodically parses new users.
- Parameters:
user (string) – Name of the user to register.
directory (pathlib.Path) – Dedicated folder that the BOINC server can reach (i.e. on CERN AFS or EOS), which will hold the new submissions and results. Should not be accessed manually by the user to avoid syncing issues.
- Return type:
None.
- class ResultRetriever(user, dev_server=False, silent=False)[source]
Class to retrieve and manage results from Xboinc simulations.
This class provides functionality to retrieve, index, and manage simulation results from BOINC work units. It can untar result files, create indexes, and provide various views and statistics about completed jobs.
- _directory
Path to the directory containing results
- Type:
FsPath
- _df
Indexed DataFrame of all available results
- Type:
pd.DataFrame
Examples
>>> retriever = ResultRetriever('myuser', dev_server=True) >>> studies = retriever.get_study_list() >>> for job_name, particles in retriever.iterate_results('my_study'): ... # Process particles data ... pass
- _untar_results(path: xaux.FsPath, silent: bool = False)[source]
Untar all compressed result files in the given path.
- Parameters:
path (FsPath) – Directory path containing .tar.gz files to extract
silent (bool, optional) – If True, suppress progress bar output (default: False)
- _index_results(path: xaux.FsPath, silent: bool = False) pandas.DataFrame[source]
Index all result files in the given path and create a DataFrame.
Scans for .bin files in subdirectories and extracts metadata from filenames to create a structured index of available results.
- Parameters:
path (FsPath) – Directory path to scan for result files
silent (bool, optional) – If True, suppress progress bar output (default: False)
- Returns:
DataFrame with columns: user, study_name, job_name, wu_name, bin_file Each row represents one available result file
- Return type:
pd.DataFrame
- _user
- _domain
- _dev_server = False
- _to_delete = []
- _df
- get_overview()[source]
Get a comprehensive overview of all available results.
- Returns:
DataFrame containing all indexed results with columns: user, study_name, job_name, wu_name, bin_file
- Return type:
pd.DataFrame
- get_study_status(study_name, verbose=False)[source]
Get detailed status information for a specific study.
Compares local results with server work units to provide comprehensive status information including completion rates and missing jobs.
- Parameters:
- Returns:
list: Job names available in results
set: Job names missing from results but present on server
- Return type:
- Raises:
ValueError – If study_name is not found in results or server work units
Warning
- UserWarning
If there are mismatches between local results and server status
- iterate_results(study_name)[source]
Iterate over all results for a specific study.
Yields tuples of job names and their corresponding particle data for all completed jobs in the specified study.
- Parameters:
study_name (str) – Name of the study to iterate over
- Yields:
tuple of (str, xpart.Particles) – Job name and corresponding particles object for each result
- Raises:
ValueError – If study_name is not found in available results
Warning
- UserWarning
If a binary file is incompatible with current Xboinc version
Examples
>>> retriever = ResultRetriever('myuser', dev_server=True) >>> for job_name, particles in retriever.iterate_results('my_study'): ... print(f"Processing job: {job_name}") ... print(f"Number of particles: {len(particles.x)}")
- clean(study_name)[source]
Clean up results for a specific study.
Removes all binary result files, empty directories, and clears the study from the internal DataFrame index.
- Parameters:
study_name (str) – Name of the study to clean up
- Raises:
ValueError – If study_name is not found in available results
Warning
This operation is irreversible. All result files for the study will be permanently deleted.
- classmethod iterate(user, study_name, dev_server=False, silent=False)[source]
Class method to directly iterate over results for a user and study.
Convenient method that creates a ResultRetriever instance and immediately starts iterating over results without requiring explicit instantiation.
- Parameters:
- Yields:
tuple of (str, xpart.Particles) – Job name and corresponding particles object for each result
Examples
>>> for job_name, particles in ResultRetriever.iterate('myuser', 'my_study', dev_server=True): ... # Process particles data ... pass
- classmethod overview(user, dev_server=False, silent=False)[source]
Class method to get an overview of results for a specific user.
- Parameters:
- Returns:
DataFrame with overview of all available results
- Return type:
pd.DataFrame
Examples
>>> overview_df = ResultRetriever.overview('myuser', dev_server=True) >>> print(overview_df.groupby('study_name').size())
- classmethod status(user, study_name, dev_server=False, silent=False, verbose=False)[source]
Class method to get status of results for a specific user and study.
- Parameters:
user (str) – The user that submitted the BOINC jobs
study_name (str) – Name of the study to check status for
dev_server (bool, optional) – Whether to use development server (default: False)
silent (bool, optional) – Whether to suppress output messages (default: True)
verbose (bool, optional) – If True, print detailed job lists (default: False)
- Returns:
list: Job names available in results
set: Job names missing from results but present on server
- Return type:
Examples
>>> available, missing = ResultRetriever.status('myuser', 'my_study', dev_server=True) >>> print(f"Available jobs: {len(available)}, Missing jobs: {len(missing)}")
- classmethod study_list(user, dev_server=False, silent=False)[source]
Class method to get a list of all studies for a specific user.
- Parameters:
- Returns:
Sorted list of unique study names found in the results
- Return type:
Examples
>>> studies = ResultRetriever.study_list('myuser', dev_server=True) >>> print(studies)
- class XbInput(**kwargs)[source]
Bases:
xobjects.Struct- _version
- num_turns
- num_elements
- ele_start
- ele_stop
- checkpoint_every
- _parity_check
- xb_state
- line_metadata
- classmethod from_binary(filename, offset=0, raise_version_error=True)[source]
Create an XbInput from a binary file. The file should not contain anything else (otherwise the offset will be wrong).
- Parameters:
filename (pathlib.Path) – The binary containing the simulation state.
- Return type:
- to_binary(filename)[source]
Dump the XbInput to a binary file.
- Parameters:
filename (pathlib.Path) – The binary containing the simulation state.
- Return type:
None.
- property version
- property line
- property particles
- class XbState(**kwargs)[source]
Bases:
xobjects.Struct- _version
- _i_turn
- _xsize
- _particles
- classmethod from_binary(filename, offset=0, raise_version_error=True)[source]
Create an XbState from a binary file. The file should not contain anything else (otherwise the offset will be wrong).
- Parameters:
filename (pathlib.Path) – The binary containing the simulation state.
- Return type:
- to_binary(filename)[source]
Dump the XbState to a binary file.
- Parameters:
filename (pathlib.Path) – The binary containing the simulation state.
- Return type:
None.
- property version
- property particles
- property i_turn
- class JobManager(user, study_name, line=None, dev_server=False, **kwargs)[source]
A class to manage jobs for submission to the Xboinc server.
This class provides a convenient interface for adding multiple particle tracking jobs and submitting them as a batch to the BOINC server. It handles job validation, time estimation, file preparation, and submission.
The JobManager ensures that: - Job execution times fall within acceptable bounds - Job names are unique within a study - All necessary files are properly packaged and submitted - Proper cleanup is performed after submission
Examples
Basic usage with a single line for all jobs:
>>> line = xtrack.Line.from_dict(line_dict) >>> manager = JobManager("user123", "my_study", line=line, dev_server=True) >>> manager.add(job_name="job1", num_turns=1000, particles=particles1) >>> manager.add(job_name="job2", num_turns=2000, particles=particles2) >>> manager.submit()
Usage with different lines per job:
>>> manager = JobManager("user123", "my_study", dev_server=True) >>> manager.add(job_name="job1", num_turns=1000, particles=particles1, line=line1) >>> manager.add(job_name="job2", num_turns=2000, particles=particles2, line=line2) >>> manager.submit()
- dev_server = False
- _user
- _domain
- _study_name
- _line = None
- _submit_file = 'Uninferable__Uninferable__Uninferable.tar.gz'
- _json_files = []
- _bin_files = []
- _tempdir
- _submitted = False
- _unique_job_names
- _assert_not_submitted()[source]
Ensure that jobs have not already been submitted.
- Raises:
ValueError – If jobs have already been submitted from this JobManager instance.
- add(*, job_name, num_turns, ele_start=0, ele_stop=-1, particles, line=None, checkpoint_every=-1, **kwargs)[source]
Add a single job to the JobManager instance.
This method creates the necessary input files (binary and JSON metadata) for a single tracking job. The job is validated for execution time bounds and prepared for batch submission.
- Parameters:
job_name (str) – Unique name for this job within the study. Cannot contain ‘__’ (double underscore). If a duplicate name is provided, it will be automatically renamed with a numeric suffix.
num_turns (int) – The number of tracking turns for this job. Must be positive.
ele_start (int, optional) – The starting element index for tracking. Default is 0 (first element). If provided different from 0 with particles set at a certain starting position, raises a ValueError.
ele_stop (int, optional) – The stopping element index for tracking. Default is -1 (last element).
particles (xpart.Particles) – The particles object containing the initial particle distribution to be tracked.
line (xtrack.Line, optional) – The tracking line for this specific job. If None, uses the line provided during JobManager initialization. Providing a line per job is slower due to repeated preprocessing.
checkpoint_every (int, optional) – Checkpoint interval in turns. Default is -1 (no checkpointing). If positive, simulation state will be saved every N turns.
**kwargs – Additional job metadata to be included in the job JSON file.
- Raises:
ValueError – If job_name contains ‘__’, if no line is available, or if the estimated execution time is outside acceptable bounds.
Notes
Job execution time is estimated using benchmark data and must fall between LOWER_TIME_BOUND (90s) and UPPER_TIME_BOUND (3 days).
The method creates two files per job: - A .json file with job metadata - A .bin file with the binary simulation input data
Examples
>>> manager.add( ... job_name="scan_point_1", ... num_turns=10000, ... particles=my_particles, ... custom_param=42 ... )
- submit()[source]
Package and submit all added jobs to the BOINC server.
This method creates a compressed tar archive containing all job files and moves it to the user’s submission directory where the BOINC server will periodically check for new submissions.
The submission process: 1. Creates a .tar.gz archive with all job files 2. Moves the archive to the appropriate submission directory 3. Cleans up temporary files 4. Marks the JobManager as submitted
- Raises:
ValueError – If jobs have already been submitted or if the user domain is invalid.
Notes
After submission, this JobManager instance cannot be used to add more jobs. Create a new JobManager instance for additional submissions.
The submission directory depends on the dev_server setting: - Development server: {user_directory}/input_dev - Production server: {user_directory}/input
Examples
>>> manager.submit() Zipping files: 100%|██████████| 4/4 [00:00<00:00, 1234.56it/s] Submitted 2 jobs to BOINC server for user user123 in study my_study.
- __len__()[source]
Return the number of jobs added to this JobManager instance.
- Returns:
The number of jobs that have been added but not yet submitted.
- Return type:
- __repr__()[source]
Return a string representation of the JobManager instance.
- Returns:
A concise string representation showing key attributes and status.
- Return type:
Examples
>>> manager = JobManager("user123", "my_study", dev_server=True) >>> repr(manager) 'JobManager(user=user123, study_name=my_study, num_jobs=0, dev_server=True, submitted=False)'
- get_job_summary()[source]
Return a comprehensive summary of all jobs in this JobManager instance.
- Returns:
A dictionary containing: - user: The submitting user - study_name: The study name - num_jobs: Total number of jobs - dev_server: Whether using development server - jobs: List of individual job summaries with name, turns, and particle count - submitted: Whether jobs have been submitted
- Return type:
Examples
>>> summary = manager.get_job_summary() >>> print(f"Study {summary['study_name']} has {summary['num_jobs']} jobs") >>> for job in summary['jobs']: ... print(f"Job {job['job_name']}: {job['num_particles']} particles, {job['num_turns']} turns")