Description

This class maps sub-domains of the global simulation domain to each MPI rank.

The global domain is split along the longest axis. Within each sub-domain, there are layers of ownership:

0 < RANK < NUM_RANKS - 1:

Unowned_up (high + ghostlayer <= pos) Ghost_up (high <= pos < high + ghostlayer) Shared_up (high - ghostlayer <= pos < high) Owned (low + ghostlayer <= pos < high - ghostlayer) Shared_down (low <= pos < low + ghostlayer) Ghost_down (low - ghost_layer <= pos < low) Unowned_down (pos < low - ghostlayer)

RANK = 0:

Unowned_up (high + ghostlayer <= pos) Ghost_up (high <= pos < high + ghostlayer) Shared_up (high - ghostlayer <= pos < high) Owned (low <= pos < high - ghostlayer) Unowned_down (pos < low)

RANK = NUM_RANKS - 1:

Unowned_up (high <= pos) Owned (low + ghostlayer <= pos < high) Shared_down (low <= pos < low + ghostlayer) Ghost_down (low - ghost_layer <= pos < low) Unowned_down (pos < low - ghostlayer)

At AddBody: ** Unowned_up/Unowned_down: Bodies in this region do not interact with this rank ** Ghost_up/Ghost_down: Bodies in these regions are added to this rank, and expect to be updated by a neighbor rank every timestep ** Shared_up/Shared_down: Bodies in these regions are added to this rank, and send updates to a neighbor rank every timestep ** Owned: Bodies in this region are added to this rank only and have no interaction with other ranks.

Mid-Simulation: ** Unowned_up/Unowned_down: Bodies in these regions do not interact with this rank ** Ghost_up/Ghost_down: –If a body in one of these regions was most recently in this rank, it will be simulated on this rank and sent to update a ghost body on a neighbor rank every timestep. –If a body in one of these regions was most recently in another rank, it will be simulated on this rank and will be updated by a neighbor rank every timestep. ** Shared_up/Shared_down: –If a body in one of these regions was most recently in this rank, it will be simulated on this rank and sent to update a ghost body on a neighbor rank every timestep. –If a body in one of these regions was most recently in another rank, it will be simulated on this rank and will be updated by a neighbor rank every timestep. ** Owned: Bodies in this region are simulated only on this rank.

Actions:

A body with an OWNED comm_status will be packed for exchange to create a ghost body on another rank when it passes into one of this rank's shared regions, at which point the body is given a SHARED comm_status on this rank.

A body with a SHARED comm_status will become OWNED when it moves into the owned region on this rank. A body with a SHARED comm_status will become GHOST when it moves outside of [sublo, subhi), the other rank updates its comm_status for that body to SHARED and takes charge of updating the body.

A body with a GHOST comm_status will become OWNED when it moves into the owned region of this rank. A body with a GHOST comm_status will be removed when it moves into the one of this rank's unowned regions.

#include <ChDomainDistributed.h>

Collaboration diagram for chrono::ChDomainDistributed:

Public Member Functions

 ChDomainDistributed (ChSystemDistributed *sys)
 
void SetSimDomain (double xlo, double xhi, double ylo, double yhi, double zlo, double zhi)
 Defines the global space used for simulation. More...
 
virtual distributed::COMM_STATUS GetBodyRegion (int index)
 Return the location of the specified body within this rank based on the data manager.
 
virtual distributed::COMM_STATUS GetBodyRegion (std::shared_ptr< ChBody > body)
 Returns the location of the specified body within this rank based on the body-list.
 
ChVector< double > GetBoxLo ()
 Get the lower bounds of the global simulation domain.
 
ChVector< double > GetBoxHi ()
 Get the upper bounds of the global simulation domain.
 
ChVector< double > GetSubLo ()
 Get the lower bounds of the local sub-domain.
 
ChVector< double > GetSubHi ()
 Get the upper bounds of the local sub-domain.
 
void SetSplitAxis (int i)
 Sets the axis along which the domain will be split x=0, y=1, z=2.
 
int GetSplitAxis ()
 x = 0, y = 1, z = 2
 
int GetRank (ChVector< double > pos)
 Returns the rank which has ownership of a body with the given position.
 
bool IsSplit ()
 Returns true if the domain has been set.
 
virtual void PrintDomain ()
 Prints basic information about the domain decomposition.
 

Public Attributes

ChVector< double > boxlo
 Lower coordinates of the global domain.
 
ChVector< double > boxhi
 Upper coordinates of the global domain.
 
ChVector< double > sublo
 Lower coordinates of this subdomain.
 
ChVector< double > subhi
 Upper coordinates of this subdomain.
 

Protected Member Functions

virtual void SplitDomain ()
 Divides the domain into equal-volume, orthogonal, axis-aligned regions along the longest axis. More...
 

Protected Attributes

ChSystemDistributedmy_sys
 
int split_axis
 Index of the dimension of the longest edge of the global domain.
 
bool split
 Flag indicating that the domain has been divided into sub-domains.
 
bool axis_set
 Flag indicating that the splitting axis has been set.
 

Member Function Documentation

◆ SetSimDomain()

void ChDomainDistributed::SetSimDomain ( double  xlo,
double  xhi,
double  ylo,
double  yhi,
double  zlo,
double  zhi 
)

Defines the global space used for simulation.

This space cannot be changed once set and needs to encompass all of the possible simulation region. If a body leaves the specified simulation domain, it may be removed from the simulation entirely.

◆ SplitDomain()

void ChDomainDistributed::SplitDomain ( )
protectedvirtual

Divides the domain into equal-volume, orthogonal, axis-aligned regions along the longest axis.

Needs to be called right after the system is created so that bodies are added correctly.


The documentation for this class was generated from the following files:
  • /builds/uwsbel/chrono/src/chrono_distributed/physics/ChDomainDistributed.h
  • /builds/uwsbel/chrono/src/chrono_distributed/physics/ChDomainDistributed.cpp