pymarine.utils package
Submodules
pymarine.utils.coordinate_transformations module
Definition of some common coordinate system conversions
Notes
The coordinate transformations defined in this section are a selection of the coordinate_transformation implemented in the astropysics model. The reason why this module is not installed by it self is that this module has not been updated for Python 3. It is therefore easier just to take the part of the module which is relevant for HMC
The definitions of the spherical coordinates follow the physics convention as described here
References
- pymarine.utils.coordinate_transformations.cartesian_to_cylindrical(x, y, z, degrees=False)[source]
Converts three arrays in 3D rectangular Cartesian coordinates to cylindrical polar coordinates.
- Parameters:
- Returns:
Cylindrical coordinates as a (rho,theta,z), with theta increasing from +x to +y and theta = 0 at x-axis.
- Return type:
tuple of 3 floats or ndarrays
- pymarine.utils.coordinate_transformations.cartesian_to_polar(x, y, degrees=False)[source]
Converts arrays in 2D rectangular Cartesian coordinates to polar coordinates.
- Parameters:
- Returns:
Polar coordinates (r, theta) where theta is measured from the +x axis increasing towards the +y axis
- Return type:
tuple of 2 floats or ndarrays
- pymarine.utils.coordinate_transformations.cartesian_to_spherical(x, y, z, degrees=False)[source]
Converts three arrays in 3D rectangular cartesian coordinates to spherical polar coordinates.
Note that the spherical coordinates are in physicist convention such that (1,pi/2,0) is x-axis.
- Parameters:
- Returns:
Spherical coordinates (r,theta,phi)
- Return type:
tuple of 3 floats or ndarrays
Examples
Convert the coordinates (1, 0, 0)
>>> radius, theta, phi = cartesian_to_spherical(1, 0, 0) >>> print("Radius = {:.4g} Theta = {:.4g} Phi = {:.4g}".format(radius, theta, phi)) Radius = 1 Theta = 1.571 Phi = 0
>>> radius, theta, phi = cartesian_to_spherical(1, 1, 1, degrees=True) >>> print("Radius = {:.4g} Theta = {:.4g} Phi = {:.4g}".format(radius, theta, phi)) Radius = 1.732 Theta = 54.74 Phi = 45
Convert array_like coordinates
>>> xx, yy, zz = [0, 0, 1 ], [1, 0, 1 ], [0, 1, 1 ] >>> radii, thetas, phis = cartesian_to_spherical(xx, yy, zz) >>> for ii, radius in enumerate(radii): ... print("({:.2f}, {:.2f} {:.2f}) -> ({:.4f}, {:.4f}, {:.4f})" ... "".format(xx[ii], yy[ii], zz[ii], radius, thetas[ii], phis[ii])) (0.00, 1.00 0.00) -> (1.0000, 1.5708, 1.5708) (0.00, 0.00 1.00) -> (1.0000, 0.0000, 0.0000) (1.00, 1.00 1.00) -> (1.7321, 0.9553, 0.7854)
References
- pymarine.utils.coordinate_transformations.colatitude_to_latitude(theta, degrees=False)[source]
Converts from colatitude/inclination (i.e. “theta” in physicist convention) to latitude (i.e. 0 at the equator).
- Parameters:
- Returns:
Latitude
- Return type:
float or ndarray
Examples
>>> colatitude_to_latitude(15, degrees=True) 75 >>> phi_list = [np.pi / i for i in range(1,6)] >>> latitude = colatitude_to_latitude(phi_list) >>> for ii, phi in enumerate(phi_list): ... print("{:6.4f} -> {:7.4f}".format(phi, latitude[ii])) 3.1416 -> -1.5708 1.5708 -> 0.0000 1.0472 -> 0.5236 0.7854 -> 0.7854 0.6283 -> 0.9425
- pymarine.utils.coordinate_transformations.cylindrical_to_cartesian(s, t, z, degrees=False)[source]
Converts three arrays in cylindrical polar coordinates to 3D rectangular Cartesian coordinates.
- Parameters:
- Returns:
Cartesian coordinates as an (x,y,z) tuple.
- Return type:
tuple of 3 floats or ndarrays
- pymarine.utils.coordinate_transformations.latitude_to_colatitude(lat, degrees=False)[source]
Converts from latitude (i.e. 0 at the equator) to colatitude/inclination (i.e. “theta” in physicist convention).
- Parameters:
- Returns:
Colatitude
- Return type:
float or ndarray
Examples
>>> latitude_to_colatitude(10, degrees=True) 80 >>> phi_list = [np.pi / i for i in range(1,6)] >>> co_latitude = latitude_to_colatitude(phi_list) >>> for ii, phi in enumerate(phi_list): ... print("{:6.4f} -> {:7.4f}".format(phi, co_latitude[ii])) 3.1416 -> -1.5708 1.5708 -> 0.0000 1.0472 -> 0.5236 0.7854 -> 0.7854 0.6283 -> 0.9425
- pymarine.utils.coordinate_transformations.polar_to_cartesian(r, t, degrees=False)[source]
Converts arrays in 2D polar coordinates to rectangular cartesian coordinates.
- Parameters:
- Returns:
Cartesian coordinates (x,y)
- Return type:
tuple of 2 floats or ndarrays
- pymarine.utils.coordinate_transformations.spherical_to_cartesian(r, t, p, degrees=False)[source]
Converts arrays in 3D spherical polar coordinates to rectangular cartesian coordinates.
- Parameters:
- Returns:
Cartesian coordinates (x,y,z)
- Return type:
tuple of 3 floats or ndarrays
Notes
The spherical coordinates are in physicist convention such that (1,pi/2,0) is the x-axis.
Examples
>>> xx, yy, zz = spherical_to_cartesian(1, 0, 0) >>> print("xx = {:.4g} yy = {:.4g} zz = {:.4g}".format(xx, yy, zz)) xx = 0 yy = 0 zz = 1 >>> xx, yy, zz = spherical_to_cartesian(1.7321, 54.736, 45, degrees=True) >>> print("xx = {:.4g} yy = {:.4g} zz = {:.4g}".format(xx, yy, zz)) xx = 1 yy = 1 zz = 1 >>> rr, tt, pp = np.array([1, 2, 3 ]), np.array([0, 45, 90 ]), np.array([0, -45, 45 ]) >>> xx, yy, zz = spherical_to_cartesian(rr, tt, pp, degrees=True) >>> for ii, x in enumerate(xx): ... print("({:6.2f}, {:6.2f} {:6.2f}) -> ({:6.2f}, {:6.2f}, {:6.2f})" ... "".format(rr[ii], tt[ii], pp[ii], x, yy[ii], zz[ii])) ( 1.00, 0.00 0.00) -> ( 0.00, 0.00, 1.00) ( 2.00, 45.00 -45.00) -> ( 1.00, -1.00, 1.41) ( 3.00, 90.00 45.00) -> ( 2.12, 2.12, 0.00)
pymarine.utils.date_time module
Several date/time helper functions.
- pymarine.utils.date_time.matlabnum2date(x)[source]
Convert a Matlab numerical date/time representation into a Python datetime object
- Parameters:
x (float or ndarrray) – Matlab numerical date/time representation giving the number of days since 0000 00 00:00:00
- Returns:
number_of_days – Date/time corresponding to the float value x
- Return type:
ndarray or scalar of the type datetime`
Notes
In Matlab, the numerical date/time representation gives the number of days since Jan 00 0000 00:00:00
In Python, the numerical date/time representation gives the number of days since Jan 01 0001 00:00:00
To convert from matlab to python, 366 days need to be subtracted from the matlab numerical date/time representation to get the Python numerical data/time representation (note that matlab starts at Jan 0th!)
Examples
The following two dates in numerical representation are obtained using the Matlab datenum function
2012-12-21T12:12:12 -> 7.352245084722222e+05 # days since 0000 00 00:00:00
1973-11-12T09:15:43 -> 7.209403859143519e+05 # days since 0000 00 00:00:00
>>> num_date_matlab = np.array([7.352245084722222222222222e+05, ... 7.209403859143519e+05])
>>> matlabnum2date(num_date_matlab) array([datetime.datetime(2012, 12, 21, 12, 12, 12, 3), datetime.datetime(1973, 11, 12, 9, 15, 43, 5)], dtype=object)
- pymarine.utils.date_time.valid_date(s)[source]
Check if supplied data s is a valid date for the format Year-Month-Day
- Parameters:
s (str) – A valid date in the form of YYYY-MM-DD, so first the year, then the month, then the day
- Returns:
Date object with the year, month, day obtained from the valid string representation
- Return type:
datetime
- Raises:
argparse.ArgumentTypeError: –
Notes
This is a helper function for the argument parser module argparse which allows you to check if the argument passed on the command line is a valid date.
Examples
This is the direct usage of valid_date to see if the date supplied is of format YYYY-MM-DD
>>> try: ... date = valid_date("1973-11-12") ... except argparse.ArgumentTypeError: ... print("This date is invalid") ... else: ... print("This date is valid") This date is valid
In case an invalid date is supplied
>>> try: ... date = valid_date("1973-15-12") ... except argparse.ArgumentTypeError: ... print("This date is invalid") ... else: ... print("This date is valid") This date is invalid
Here it is demonstrated how to add a ‘–startdate’ command line option to the argparse parser which checks if a valid date is supplied
>>> parser = argparse.ArgumentParser() >>> p = parser.add_argument("--startdate", ... help="The Start Date - format YYYY-MM-DD ", ... required=True, ... type=valid_date)
References
- https://stackoverflow.com/questions/25470844/
specify-format-for-input-arguments-argparse-python
pymarine.utils.file_and_directory module
Collection of functions dealing with the system file and directories
- pymarine.utils.file_and_directory.get_path_depth(path_name)[source]
Get the depth of a path or file name
- Parameters:
path_name (str) – Path name to get the depth from
- Returns:
depth of the path
- Return type:
Examples
>>> get_path_depth("C:\Anaconda") 1 >>> get_path_depth("C:\Anaconda\share") 2 >>> get_path_depth("C:\Anaconda\share\pywafo") 3 >>> get_path_depth(".\imaginary\path\subdir\share") 4
- pymarine.utils.file_and_directory.make_directory(directory)[source]
Create a directory in case it does not yet exist.
- Parameters:
directory (str) – Name of the directory to create
Notes
This function is used to create directories without checking if it already exists. If the directory already exists, we can silently continue.
- Raises:
OSError – The OSError is only raised if it is not an EEXIST error. This implies that the creation of the directory failed due to another reason then the directory already being present. It could be that the file system is full or that we may not have write permission
- pymarine.utils.file_and_directory.scan_base_directory(walk_dir='.', supplied_file_list=None, file_has_string_pattern='', file_has_not_string_pattern='', dir_has_string_pattern='', dir_has_not_string_pattern='', start_date_time=None, end_date_time=None, time_zone=None, time_stamp_year_first=True, time_stamp_day_first=False, extension=None, max_depth=None, sort_file_base_names=False)[source]
Recursively scan the directory walk_dir and get all files underneath obeying the search strings and/or date/time ranges
- Parameters:
walk_dir (str, optional) – The base directory to start the import. Default = “.”
supplied_file_list (list, optional) – In case walk dir is not given we can explicitly pass a file list to analyze. Default = None
dir_has_string_pattern (str, optional) – Requires the directory name to have this pattern (Default value = “”). This selection is only made on the first directory level below the walk_dir
dir_has_not_string_pattern (str, optional) – Requires the directory name NOT to have this pattern (Default value = “”). This selection is only made on the first directory level below the walk_dir
file_has_string_pattern (str, optional) – Requires the file name to have this pattern (Default value = “”, i.e., matches all)
file_has_not_string_pattern (str, optional) – Requires the file name NOT to have this pattern (Default value = “”)
extension (str or None, optional) – Extension of the file to match. If None, also matches. Default = None
max_depth (int, optional) – Sets a maximum depth to which the search is carried out. Default = None, which does not limit the search depth. For deep file structures setting a limit to the search depth speeds up the search.
sort_file_base_names (bool, option) – If True, sort the resulting file list alphabetically based on the file base name. Default = False
start_date_time (DateTime or None, optional) – If given, get the date time from the current file name and only add the files with a date/time equal or large the start_date_time. Default is None
end_date_time (DateTime or None, optional) – If given, get the date time from the current file name and only add the files with a date/time smaller than the end_date_time. Default is None
time_zone (str or None, optional) – If given add this time zone to the file stamp. The start and end time should also have a time zone
time_stamp_year_first (bool, optional) – Passed to the datetime parser. If true, the year is first in the date/time string. Default = True
time_stamp_day_first (bool, optional) – Passed to the datetime parser. If true, the day is first in the date/time string. Default = False
- Returns:
All the file names found below the input directory walk_dir obeying all the search strings
- Return type:
Examples
Find all the python files under the share directory in the Anaconda installation folder
>>> scan_dir = "C:\Anaconda\share" >>> file_list = scan_base_directory(scan_dir, extension='.py')
Find all the python files under the share directory in the Anaconda installation folder belonging to the pywafo directory
>>> file_list = scan_base_directory(scan_dir, extension='.py', ... dir_has_string_pattern="wafo")
Note that wafo matches on the directory ‘pywafo’, which is the first directory level below the scan directory. However, if we would match on ‘^wafo’ the returned list would be empty as the directory has to start with wafo.
To get all the files with “test” in the name with a directory depth smaller than three do:
>>> file_list = scan_base_directory(scan_dir, extension='.py', ... dir_has_string_pattern="wafo", ... file_has_string_pattern="test", max_depth=3)
Test the date/time boundaries. First create a file list from 28 sep 2017 00:00 to 5:00 with a hour interval and convert it to a string list
>>> file_names = ["AMS_{}.mdf".format(dt.strftime("%y%m%dT%H%M%S")) for dt in ... pd.date_range("20170928T000000", "20170928T030000", freq="30min")] >>> for file_name in file_names: ... print(file_name) AMS_170928T000000.mdf AMS_170928T003000.mdf AMS_170928T010000.mdf AMS_170928T013000.mdf AMS_170928T020000.mdf AMS_170928T023000.mdf AMS_170928T030000.mdf
Use the scan_base_directory to get the files within a specific date/time range
>>> file_selection = scan_base_directory(supplied_file_list=file_names, ... start_date_time="20170928T010000", end_date_time="20170928T023000")
>>> for file_name in file_selection: ... print(file_name) AMS_170928T010000.mdf AMS_170928T013000.mdf AMS_170928T020000.mdf
Note that the selected range run from 1 am until 2 am; the end_date_time of 2.30 am is not included
pymarine.utils.geographic module
Collection of functions dealing with geographical coordinates based on the LatLon module.
- class pymarine.utils.geographic.LocationCheck(longitude, latitude, distance_deviation_allowed=10)[source]
Bases:
objectClass to store a location in lat/lon and check if the distance of another location is out of range
- Parameters:
Notes
Use this class to store a reference location and check later if the distance of another location is out of range given by the maximum distance_deviation_allowed input argument.
Class uses the LatLon package so distances are all in km
- out_of_range(current_latitude, current_longitude)[source]
Check if the current location is outside the given distance from the target location
- Parameters:
- Returns:
True in case the current location is outside the distance from the target given by distance_deviation_allowed
- Return type:
Examples
Initialise the location_check with a target location (52.37, 4.89). The we can use the method out_of_range to test if the current_location is out of range of the target location
>>> location_check = LocationCheck(latitude=52.37, longitude=4.89, ... distance_deviation_allowed=10) >>> out_range = location_check.out_of_range(current_latitude=52.37, current_longitude=4.89) >>> print("Distance : {:.1f} km. Out of range : {}".format(location_check.distance, out_range)) Distance : 0.0 km. Out of range : False >>> out_range = location_check.out_of_range(current_latitude=52.31, current_longitude=4.84) >>> print("Distance : {:.1f} km. Out of range : {}".format(location_check.distance, out_range)) Distance : 7.5 km. Out of range : False >>> out_range = location_check.out_of_range(current_latitude=52.36, current_longitude=4.90) >>> print("Distance : {:.1f} km. Out of range : {}".format(location_check.distance, out_range)) Distance : 1.3 km. Out of range : False >>> out_range = location_check.out_of_range(current_latitude=52.31, current_longitude=4.77) >>> print("Distance : {:.1f} km. Out of range : {}".format(location_check.distance, out_range)) Distance : 10.6 km. Out of range : True
Also check around the 0th and 180th meridians
>>> greenwich = LocationCheck(latitude=51.48, longitude=-1, ... distance_deviation_allowed=10) >>> out_range = greenwich.out_of_range(current_latitude=51.48, current_longitude=1) >>> print("Distance : {:.1f} km. Out of range : {}".format(greenwich.distance, out_range)) Distance : 138.9 km. Out of range : True >>> out_range = greenwich.out_of_range(current_latitude=51.48, current_longitude=359) >>> print("Distance : {:.1f} km. Out of range : {}".format(greenwich.distance, out_range)) Distance : 0.0 km. Out of range : False
>>> pacific_ocean = LocationCheck(latitude=0.0, longitude=179, ... distance_deviation_allowed=223) >>> out_range = pacific_ocean.out_of_range(current_latitude=0.0, current_longitude=-179) >>> print("Distance : {:.1f} km. Out of range : {}".format(pacific_ocean.distance, out_range)) Distance : 222.6 km. Out of range : False >>> out_range = pacific_ocean.out_of_range(current_latitude=0.0, current_longitude=181) >>> print("Distance : {:.1f} km. Out of range : {}".format(pacific_ocean.distance, out_range)) Distance : 222.6 km. Out of range : False
Notes
The outcome of this routine has been checked with the Google Earth ruler
- pymarine.utils.geographic.get_speed_from_distance_and_time(trajectory, distance_name='travel_distance', speed_name='speed_sustained', datetime_name='DateTime', travel_time_name='travel_time', speed_max_clip=None)[source]
Calculate the speed based on the travel distance and travel time.
- Parameters:
trajectory (dataframe) – Trajectory
distance_name (str, optional) – Name of the distance column, Default value = “travel_distance”
speed_name (str, optional) – Name of the column with the sustained speed, Default value = “speed_sustained”
datetime_name (str, optional) – Name of the column with the date time, Default value = “DateTime”
distance_name – Name of the column with the travel distance, Default value = “travel_distance
travel_time_name (str, optional) – Name of the newly created column with the travel time, Default value = “travel_time”
speed_max_clip (float, optional) – clip all values above this speed in knots. Default value = None, which means that the speed is not clipped
- Returns:
Update of the input DataFrame trajectory with an extra column named speed_name containing the calculated speed
- Return type:
DataFrame
- Raises:
KeyError: – Raised in case no travel time is available
Examples
First create a data frame with some travel distance at and interval of 3h over 12 hours.
>>> data = pd.DataFrame(index=pd.date_range(start="20160101", end="20160101T120000", freq="3h")) >>> data["travel_distance"] = np.linspace(start=0, stop=60, num=data.index.size, endpoint=True)
Now calculate the speed based on the travel distance change between each time step
>>> data = get_speed_from_distance_and_time(data) >>> data travel_distance travel_time speed_sustained 2016-01-01 00:00:00 0.0 0.0 5.0 2016-01-01 03:00:00 15.0 3.0 5.0 2016-01-01 06:00:00 30.0 6.0 5.0 2016-01-01 09:00:00 45.0 9.0 5.0 2016-01-01 12:00:00 60.0 12.0 5.0
Notes
The output of travel_distance_and_heading_from_coordinates can be used as an input for get_speed_from_distance_and_time
See also
travel_distance_and_heading_from
- pymarine.utils.geographic.import_way_points(file_name, latitude_name='latitude', longitude_name='longitude', heading_name='heading', travel_distance_name='travel_distance', n_distance_points=2000, start_wp=None)[source]
Import the way points of a Google Earth kml file.
- Parameters:
file_name (str) – Name of the google earth kml file to import
latitude_name (str, optional) – Name of the latitudes column (Default value = “latitude”)
longitude_name (str, optional) – Name of the longitudes column (Default value = “longitude”)
heading_name (str, optional) – Name of the heading column (Default value = “heading”)
travel_distance_name (str, optional) – Name of the travel distance column (Default value = “travel_distance”)
n_distance_points (int, optional) – Number of points to use to interpolate the kml data (Default value = 2000)
start_wp (int, optional) – If given, the first start_wp pins are skipped (Default value = None)
- Returns:
New data frame with the latitude and longitude coordinates. The travel distance is used for the index of the DataFrame
- Return type:
DataFrame
Notes
The kml input data can be made in Google Earth by putting a list of way points (yellow pin points) to the map and store this into one directory. Then, with the right mouse button on this directory you can choose “Save Place As” and pick the kml format.
If the n_distance_points is set to None, only the coordinates of the pin locations as found in the kml file are imported.
If n_distance_points is specified, an equidistant set of coordinates obtained from the interpolated values between the pin location in the kml file is returned.
In case that interpolation is used, make sure that there is at least one sample point in between the pin locations as defined in the kml file.
Examples
First import the pure coordinates as defined at the pin point
>>> data = import_way_points("../data/madeira_ivory.kml", n_distance_points=None) >>> data.info() <class 'pandas.core.frame.DataFrame'> Float64Index: 15 entries, 0.0 to 2519.33685128 Data columns (total 3 columns): latitude 15 non-null float64 longitude 15 non-null float64 heading 15 non-null float64 dtypes: float64(3) memory usage: 480.0 bytes
>>> data["latitude"].head() travel_distance 0.000000 32.400000 51.353449 31.800000 103.761006 31.090513 280.375912 28.358815 540.155420 24.033664 Name: latitude, dtype: float64
>>> data["longitude"].head() travel_distance 0.000000 -17.180000 51.353449 -17.900000 103.761006 -18.498083 280.375912 -19.776735 540.155420 -20.206784 Name: longitude, dtype: float64
Do the same import but interpolate in between the pin locations on a regular grid
>>> data = import_way_points("../data/madeira_ivory.kml", n_distance_points=2000) >>> data["latitude"].head() travel_distance 0.000000 32.400000 2.441572 32.371429 3.662502 32.357143 4.883529 32.342857 6.104652 32.328571 Name: latitude, dtype: float64
>>> data["longitude"].head() travel_distance 0.000000 -17.180000 2.441572 -17.214286 3.662502 -17.231429 4.883529 -17.248571 6.104652 -17.265714 Name: longitude, dtype: float64
- pymarine.utils.geographic.travel_distance_and_heading_from_coordinates(db, latitude_name='GPS_LATITUDE', longitude_name='GPS_LONGITUDE', heading_name='HEADING', travel_distance_name='travel_distance')[source]
Calculate the travel distance and optionally the heading based on the latitude and longitude columns in the DataFrame db
- Parameters:
db (DataFrame) – Data base carrying all the data (including the latitude and longitude)
latitude_name (str, optional) – Name of the latitude column (Default value = “GPS_LATITUDE”)
longitude_name (str, optional) – Name of the longitude column (Default value = “GPS_LONGITUDE”)
heading_name (str, optional) – The heading name. It is assumed that the heading entry already exist and that only the missing values are added based on the coordinate displacement (Default = “HEADING”)
travel_distance_name (str, optional) – name of the newly created column with the travel distance in nautical miles (Default value = “travel_distance”)
- Returns:
DataFrame with an extra column named travel_distance_name containing the travel distance in nautical miles and optionally an extra column named heading_name with the headings.
- Return type:
DataFrame
Examples
First create a data frame with some latitude longitude values at an 3 hour interval over 12 hour
>>> data = pd.DataFrame(index=pd.date_range(start="20160101", end="20160101T120000", freq="3h")) >>> data["GPS_LATITUDE"] = np.linspace(start=55.4, stop=54.4, num=data.index.size) >>> data["GPS_LONGITUDE"] = np.linspace(start=3.34, stop=3.14, num=data.index.size)
Now use the travel_distance_and_heading_from_coordinates function to add the travel distance and heading to the data frame. A new data frame will be copied to the output
>>> data = travel_distance_and_heading_from_coordinates(data) >>> data GPS_LATITUDE GPS_LONGITUDE travel_distance HEADING 2016-01-01 00:00:00 55.40 3.34 0.000000 186.534194 2016-01-01 03:00:00 55.15 3.29 15.125795 186.574900 2016-01-01 06:00:00 54.90 3.24 30.252197 186.615478 2016-01-01 09:00:00 54.65 3.19 45.379211 186.655928 2016-01-01 12:00:00 54.40 3.14 60.506836 186.655928
Check the 180 meridian as well. Split linspace for the longitude in two parts in order to introduce the discontinuity in longitude when passing the 180-meridian
>>> data = pd.DataFrame(index=pd.date_range(start="20160101", end="20160101T120000", freq="3h")) >>> data["GPS_LATITUDE"] = np.linspace(start=0.0, stop=0, num=data.index.size) >>> data["GPS_LONGITUDE"] = np.append(np.linspace(start=-179.0, stop=-180, num=data.index.size//2, ... endpoint=False), ... np.linspace(start=180.0, stop=179, num=data.index.size//2+1)) >>> data = travel_distance_and_heading_from_coordinates(data) >>> data GPS_LATITUDE GPS_LONGITUDE travel_distance HEADING 2016-01-01 00:00:00 0.0 -179.0 0.000000 270.0 2016-01-01 03:00:00 0.0 -179.5 30.053858 270.0 2016-01-01 06:00:00 0.0 180.0 60.107716 270.0 2016-01-01 09:00:00 0.0 179.5 90.161575 270.0 2016-01-01 12:00:00 0.0 179.0 120.215433 270.0
Notes
The DataFrame db must have at least two columns containing the latitude and longitude. In case more columns are present this is not a problem: all columns will be copied the the output
Based on the latitude and longitude values, the travel distance and heading is calculated using the LatLon package * In case the heading field already exists, only the missing values will be updated.
The column names of the latitude, longitude, distance and headding can be defined via the arguments of the function.
See also
get_speed_from_distance_and_timecalculate the speed based on the distance and time
import_way_pointsimport a list of coordinates from a kml file generated in Google Earth
pymarine.utils.misc module
Some miscellaneous functions
- class pymarine.utils.misc.Chdir(new_path)[source]
Bases:
objectClass to move to a directory, do something, and move back when done
- Parameters:
new_path (str) – Location where you want to do something
Notes
Used on the Gompute cluster in the batch processing script to submit a job inside a directory and then move back to the higher directory to move to the next case
Examples
Go to a known directory (C:/)
>>> os.chdir("C:/") >>> os.getcwd() 'C:\\'
With the Chdir command we move to the C:/Temp directory where we can do something.
>>> with Chdir("C:/Temp") as d: ... # in this block we can do something in the directory Temp. ... os.getcwd() 'C:\\Temp'
We have left the block under Chdir, so we are back at the directory where we started
>>> os.getcwd() 'C:\\'
- class pymarine.utils.misc.ConditionalDecorator(dec, condition)[source]
Bases:
objectAdd a decorator to a function only if the condition is True
- Parameters:
dec (decorator) – The decorator which you want to add when condition is true
condition (bool) – Only add the decorator if this condition is True
- class pymarine.utils.misc.PackageInfo(module_object)[source]
Bases:
objectA class to analyse the version properties of this package
- Parameters:
module_object (
Module) – reference to the module for which want to to store the properties
- class pymarine.utils.misc.Timer(message='Elapsed time', name='routine', verbose=True, units='ms', n_digits=0, field_width=20)[source]
Bases:
objectClass to measure the time it takes execute a section of code
- Parameters:
Example
Use a with / as construction to enclose the section of code which need to be timed
>>> from numpy import allclose >>> number_of_seconds = 1.0 >>> with Timer(units="s", n_digits=0) as timer: ... time.sleep(number_of_seconds) Elapsed time routine : 1 s >>> allclose(number_of_seconds, timer.secs, rtol=0.1) True
- pymarine.utils.misc.clear_argument_list(argv)[source]
Small utility to remove the ‘\r’ character from the last argument of the argv list appearing in cygwin
- pymarine.utils.misc.clear_path(path_name)[source]
- routine to clear spurious dots and slashes from a path name
example bla/././oke becomes bla/oke
- Parameters:
path_name – return: clear_path as a string
- Returns:
clear_path as a string
- Return type:
Examples
>>> long_path = os.path.join(".", "..", "ok", "yoo", ".", ".", "") + "/" >>> print(long_path) .\..\ok\yoo\.\.\/ >>> print(clear_path(long_path)) ..\ok\yoo
- pymarine.utils.misc.compare_objects(obj1, obj2, counter=0, max_recursion_depth=4)[source]
Compare if two object are equal
- Parameters:
Notes
This function compares all the attributes of two objects to see if their values are the same
An attribute field may be another object which we also want to compare with the same attribute of the other object. This is done by recursively calling this function again.
Due to the recursive call mechanism we may end up in a infinite loop. To prevent this, a maximum recursion depth can be given.
The test function test_sequence_tool of the sequence_tool_utils module uses this function to compare to SequenceToolSummary objects
- Raises:
AssertionError: – In case on of the object fields is not equal
- pymarine.utils.misc.delete_module(modname, paranoid=None)[source]
Delete a module from memory which loaded before
- pymarine.utils.misc.get_clean_version(version)[source]
turns the full version string into a clean one without the build
- Parameters:
version (str) – The version string as return from versioneer.
- Returns:
The clean version string
- Return type:
Notes
The version string matches the following regular expression
“([.|d]+)([+]*)(.*)”
This function return the clean version string given by the part “([.|d]+)”
Examples
>>> get_clean_version("1.3") '1.3' >>> get_clean_version("2.5+dev.g43429") '2.5' >>> get_clean_version("4.3.1+dev.g43429-dirty") '4.3.1'
- pymarine.utils.misc.get_python_version_number(version_info)[source]
Script to turn the version info as obtained with sys.version_info into a digit number
- Parameters:
version_info – return: a string with the current python version as a clear digit, i.e., 3.5.3
- Returns:
a string with the current python version as a clear digit, i.e. 3.5.3
- Return type:
Examples
>>> version_string = get_python_version_number(sys.version_info)
- pymarine.utils.misc.get_regex_pattern(search_pattern)[source]
Routine to turn a string into a regular expression which can be used to match a string
- Parameters:
search_pattern (str) – A regular expression in the form of a string
- Returns:
A regular expression as return by the re.compile function or None in case a invalid regular expression was given
- Return type:
None or compiled regular expression
Notes
An empty string or an invalid search_pattern will yield a None return
- pymarine.utils.misc.get_time_stamp_from_string(string_with_date_time, yearfirst=True, dayfirst=False, timezone=None)[source]
Try to get a date/time stamp from a string
- Parameters:
string_with_date_time (str) – The string to analyses
yearfirst (bool, optional) – if true put the year first. See dateutils.parser. Default = True
dayfirst (bool, optional) – if true put the day first. See dateutils.parser. Default = False
timezone (str or None, optional) – if given try to add this time zone:w
- Returns:
Pandas data time string
- Return type:
DateTime
Examples
The date time in the file ‘AMSBALDER_160929T000000’ is 29 sep 2016 and does not have a time zone specification. The returned time stamp does also not have a time zone
>>> file_name="AMSBALDER_160929T000000" >>> time_stamp =get_time_stamp_from_string(string_with_date_time=file_name) >>> print("File name {} has time stamp {}".format(file_name, time_stamp)) File name AMSBALDER_160929T000000 has time stamp 2016-09-29 00:00:00
We can also force to add a time zone. The Etc/GMT-2 time zone is UTC + 2 time zone which is the central europe summer time (CEST) or the Europe/Amsterdam Summer time.
>>> time_stamp =get_time_stamp_from_string(string_with_date_time=file_name, ... timezone="Etc/GMT-2") >>> print("File name {} has time stamp {}".format(file_name, time_stamp)) File name AMSBALDER_160929T000000 has time stamp 2016-09-29 00:00:00+02:00
This time we assume the file name already contains a time zone, 2 hours + UTC. Since we already have a time zone, the timezone option can only convert the date time to the specified time zone.
>>> file_name="AMSBALDER_160929T000000+02" >>> time_stamp =get_time_stamp_from_string(string_with_date_time=file_name, ... timezone="Etc/GMT-2") >>> print("File name {} has time stamp {}".format(file_name, time_stamp)) File name AMSBALDER_160929T000000+02 has time stamp 2016-09-29 00:00:00+02:00
In case the time zone given by the timezone options differs with the time zone in the file name, the time zone is converted
>>> file_name="AMSBALDER_160929T000000+00" >>> time_stamp =get_time_stamp_from_string(string_with_date_time=file_name, ... timezone="Etc/GMT-2") >>> print("File name {} has time stamp {}".format(file_name, time_stamp)) File name AMSBALDER_160929T000000+00 has time stamp 2016-09-29 02:00:00+02:00
- pymarine.utils.misc.get_value_magnitude(value, convert_to_base_units=True)[source]
Get the magnitude of value with Pint dimension in terms of its base units or return a float if value does not have a dimension
- Parameters:
- Returns:
Magnitude of the value in case a Pint Quantity was added to the input or just the value itself. If convert_to_base_units was set to True the value is first converted to its SI base units
- Return type:
float or None
Examples
Assume we have a value with a pint dimension
>>> velocity = Q_("2.5 m/s") >>> print("Current velocity with dimension is: {}".format(velocity)) Current velocity with dimension is: 2.5 meter / second
We can now get the magnitude of velocity using this function as
>>> velocity_mag = get_value_magnitude(velocity) >>> print("Velocity without dimension is: {}".format(velocity_mag)) Velocity without dimension is: 2.5
In case the input argument of the get_value_magnitude is a float and does not have a dimension, the value itself is returned
>>> velocity_mag2 = get_value_magnitude(velocity_mag) >>> print("Velocity without dimension is: {}".format(velocity_mag2)) Velocity without dimension is: 2.5
In case we have a dimension in none SI units, the value is by default first converted to its SI base units.
>>> velocity_knots = Q_("1 knot") >>> velocity_mag = get_value_magnitude(velocity_knots) >>> print("Velocity {} is converted to its magnitude in m/s: {:.2f}" ... "".format(velocity_knots, velocity_mag)) Velocity 1 knot is converted to its magnitude in m/s: 0.51
In case that the convert_to_base_units flag is False we just get the magnitude in the same units as the input argument
>>> velocity_knots = Q_("2.5 knot") >>> velocity_mag = get_value_magnitude(velocity_knots, convert_to_base_units=False) >>> print("Velocity {} is converted to its magnitude in knots: {:.2f}" ... "".format(velocity_knots, velocity_mag)) Velocity 2.5 knot is converted to its magnitude in knots: 2.50
Notes
This function is used inside other functions in which it is not know before hand if an input argument is passed with or without a Pint dimension and we only are interested in the magnitude of the value. Use this function to get the magnitude
- pymarine.utils.misc.is_exe(fpath)[source]
Test if a file is an executable
- Parameters:
fpath (str) – return true or false:
- Returns:
In case fpath is a file that can be executed return True, else False
- Return type:
Notes
This function can only be used on Linux file systems as the which command is used to identity the location of the program.
- pymarine.utils.misc.move_script_path_to_back_of_search_path(script_file, append_at_the_end=True)[source]
Move the name of a script to the front or the back of the search path
- Parameters:
- Returns:
The new system path stored in a list
- Return type:
Notes
This script is sometimes required if the __version string is messing up with another __version string
Examples
sys.path = move_script_path_to_back_of_search_path(__file__)
- pymarine.utils.misc.print_banner(title, top_symbol='-', bottom_symbol=None, side_symbol=None, width=80, to_stdout=False, no_top_and_bottom=False)[source]
Create a banner for plotting a bigger title above each section in the log output
- Parameters:
title – The title to plot
top_symbol (str) – the symbol used for the top line. Default value = “-”
bottom_symbol (str) – the symbol used for the bottom line. Assume same as top if None is given (Default value = None)
side_symbol (str) – The side symbol. Assume same as top if None is given, except if top is -, then take | (Default value = None)
width (int) – the width of the banner (Default value = 80)
no_top_and_bottom (bool) – make a simple print without the top and bottom line (Default value = False)
to_stdout (bool, optional) – Print the banner to the standard output of the console instead of the logging system. Defaults to False
Examples
>>> print_banner("This is the start of a section") -------------------------------------------------------------------------------- | This is the start of a section | --------------------------------------------------------------------------------
Notes
Unless the option ‘to_stdout’ is set to True, the banner is printed via the logging system. Therefore, a logger needs to be created first
- pymarine.utils.misc.query_yes_no(question, default_answer='no')[source]
Ask a yes/no question via raw_input() and return their answer.
- pymarine.utils.misc.read_settings_file(file_name)[source]
Read the yaml file to get the setup information.
- Parameters:
file_name (str) – Name of the configuration file. Can be a full path name as well
- Returns:
All the settings as obtained from the yaml configuration file
- Return type:
Notes
The file name of the yaml file is searched for in the following order
The current directory where the script is executed. If a full path is given, this will be accepted too.
The directory where the original script is located.
In this way, a default settings file can be put in the script directory and the user does not need to copy it except a setting values needs to be changed
- Raises:
AssertionError: – In case the file cannot be found
- pymarine.utils.misc.read_value_from_dict_if_valid(dictionary, key, default_value=None)[source]
Read a value from a dictionary. If the value is not set, just return the default value
- Parameters:
dictionary – dictionary which is supposed to carry this key field
key – the name of the field to read the value from
default_value – default value in case we fail to read the key field (if it does not exist)
- Returns:
value of the key field or the default value
- Return type:
- pymarine.utils.misc.set_default_dimension(parse_value, default_dimension=None, force_default_units=False)[source]
Add a pint dimension to a value
- Parameters:
parse_value (ndarray or str or float) – Value with optional a dimension written in the form of a str. Can be an array or list of strings as well
default_dimension (str) – Required default dimension
force_default_units (bool) – If true the only allowed dimension is the default dimension. Raise an error in case this is not the case. Default = False
- Returns:
Value with the quantity as give by the default
- Return type:
Quantity- Raises:
AssertionError – In case the dimension of the parse_value argument is not not but: 1. Its dimensionality is not the same as the dimensionality of the default_dimension 2. Its units is not the same as the unit of the default_dimension and the force_default_units flag is set to True
Notes
This function is a add-on to the pint module, a package to define, operate and manipulate physical quantities: https://pypi.python.org/pypi/Pint.
This function is used to add a dimension to a value which is parsed from a text file.
It is checked if the value given in the text file has dimension already, for example that it was given as “1.0 m/s”.
If a dimension was given already: check if the dimensionality (in this case: Length/Time) is the same as the dimensionality of the default_dimension input argument.
In case the input value does not have an explicit dimension, the dimension given by default_dimension is added to the value.
This function works on both scalar and list values
Examples
Assume we want to read input values from a text file as plain numbers and we want to add a default dimension of meter to it in case the value do not have an explicit dimension yet. Just do:
>>> value_without_dimension = 1.0 # value as we read from the text file >>> value_with_dimension = set_default_dimension(value_without_dimension, "meter") >>> print(value_with_dimension) 1.0 meter
The variable value_with_dimension is now a pint quantity which carries the dimension meter.
In case the input variable already has a dimension, we should also be able to use this function. The only requirement is that the dimensionality is the same. So this should work:
>>> value_with_dimension = set_default_dimension("2.5 meter", "meter") >>> print(value_with_dimension) 2.5 meter
This should work as well
>>> value_with_dimension = set_default_dimension("5.0 mm", "meter") >>> print(value_with_dimension) 5.0 millimeter
But this fails as the dimensionality of the input argument is not corresponding with the dimensionality of the default dimension
>>> try: ... value_with_dimension = set_default_dimension("5.0 mm", "second") ... except AssertionError: ... print("This fails because the dimensionality is not the same") This fails because the dimensionality is not the same
This function should also work for arrays and list
>>> values_without_dimension = np.linspace(0, 1, num=5) >>> values_with_dimension = set_default_dimension(values_without_dimension, ... "meter/second^2") >>> print(values_with_dimension) [ 0. 0.25 0.5 0.75 1. ] meter / second ** 2
Notes
Hz are not converted to rad/s as expected. Therefore, do not try to use this to convert Hz -> rad/s
If the input argument parse_val is None, a None is returned as output as well
- pymarine.utils.misc.set_value_if_valid(value, new_value)[source]
small routine to set a value on if it is not none. Otherwise, the original value is taken
- Parameters:
value – the original value which you can pre-define with a default value
new_value – the new value. Only set this if it is not none
- Returns:
net value or the original if new_value was None
- Return type:
pymarine.utils.numerical module
Some numerical utilities used in other modules
- pymarine.utils.numerical.ecdf2percentile(ecdf, percentile)[source]
Calculate a percentile of an Empirical CDF function as returned by the statsmodels package
- Parameters:
ecdf (
statsmodel.distributions.empirical_distribution.ECDF) – Empirical CDF as a step function.percentile (float) – Percentile of the distribution
- Returns:
value where a fraction of percentile will be lower
- Return type:
Examples
>>> import statsmodels.api as sm >>> np.random.seed(0) >>> number_of_observations = 100 >>> # generate random data variing in between 0 and 100 >>> x_data = 100 * np.random.rand(number_of_observations) >>> # calculate the cumulative distribution function of this random data >>> e_cdf = sm.distributions.empirical_distribution.ECDF(x_data) >>> # print some P values belonging to the percentiles >>> print("percentile {:2.1f} : ecdf ={:6.2f}".format(0.1, ... ecdf2percentile(ecdf=e_cdf, percentile=0.1))) percentile 0.1 : ecdf = 9.61 >>> print("percentile {:2.1f} : ecdf ={:6.2f}".format(0.5, ... ecdf2percentile(ecdf=e_cdf, percentile=0.5))) percentile 0.5 : ecdf = 46.87 >>> print("percentile {:2.1f} : ecdf ={:6.2f}".format(0.9, ... ecdf2percentile(ecdf=e_cdf, percentile=0.9))) percentile 0.9 : ecdf = 89.18
- pymarine.utils.numerical.extrap1d(interpolator)[source]
Extrapolate the interp1d function outside the boundaries
Deprecated since version 0.3.4.
Deprecated function, the scipy interp1d now can extrapolate as well. This function is maintained for backward compatibility.
- Parameters:
interpolator (interp1d) – The interpolator object created with scipy
- Returns:
New interpolator object that extrapolates values outside the range
- Return type:
function
Examples
Assumed you have a x and y array you want to interpolate. You can use the scipy interp1d function for that
>>> from scipy.interpolate import interp1d
>>> xp = np.linspace(0, 3, 4) >>> yp = xp**2 >>> print(np.vstack((xp, yp))) [[ 0. 1. 2. 3.] [ 0. 1. 4. 9.]]
Use the data samples to create a Interpolator f_inter
>>> f_inter = interp1d(xp, yp)
Interpolation on a new mesh within the boundaries of the previous mesh can be done as
>>> xp_new = np.linspace(0, 3, 6) >>> yp_new = f_inter(xp_new) >>> print(np.vstack((xp_new, yp_new))) [[ 0. 0.6 1.2 1.8 2.4 3. ] [ 0. 0.6 1.6 3.4 6. 9. ]]
However, perhaps you want to extend the boundaries outside the initial boundaries. In that case you can use extrap1 in order to extrapolate the function outside the boundaries. First create the Extrapolator using the scipy Interpolator
>>> f_extra = extrap1d(f_inter)
Now we can also extrapolate outside the range
>>> xp_new2 = np.linspace(0, 4, 9) >>> yp_new2 = f_extra(xp_new2) >>> print(np.vstack((xp_new2, yp_new2))) [[ 0. 0.5 1. 1.5 2. 2.5 3. 3.5 4. ] [ 0. 0.5 1. 2.5 4. 6.5 9. 11.5 14. ]]
In the latest scipy version extrapolation is possible with the interp1d function as well :
>>> f_extra2 = interp1d(xp, yp, fill_value="extrapolate") >>> yp_new3 = f_extra2(xp_new2) >>> print(np.vstack((yp_new2, yp_new3))) [[ 0. 0.5 1. 2.5 4. 6.5 9. 11.5 14. ] [ 0. 0.5 1. 2.5 4. 6.5 9. 11.5 14. ]]
As you can see, the result is the same as the extrap1d function. The native scipy.interp1d with extrapolate as fill_values is recommended; the extrap1d function is only kept for backward compatibility and may be dropped soon
We can plot the results and compare it with the original data line
>>> import matplotlib.pyplot as plt >>> l = plt.plot(xp, yp, "o", label="samples") >>> l = plt.plot(xp_new2, yp_new2, "-x", label="extrap1d") >>> xp_new3 = np.linspace(0, 4, 50) >>> yp_new3 = xp_new3**2 >>> l = plt.plot(xp_new3, yp_new3, "-", label="original") >>> l = plt.legend() >>> plt.ion() >>> plt.show()
Notes
In case you want to use interpolate with values outside of the boundaries, this function allows to extrapolate outside the boundaries as described here interp
References
http://stackoverflow.com/questions/2745329/ how-to-make-scipy-interpolate-give-an-extrapolated-result-beyond-the-input-range https://stackoverflow.com/a/37172840/4515114
- pymarine.utils.numerical.find_idx_nearest_val(array, value)[source]
Find the nearest index of a value in a array.
- Parameters:
array (array_like) – an array with values
value (float) – the value for which we want the nearest index
- Returns:
- The index of the nearest value to ‘value’ in array.
the number of items in the array
- Return type:
Examples
>>> data_array = np.linspace(0, 10, 8, endpoint=False) >>> print(data_array) [ 0. 1.25 2.5 3.75 5. 6.25 7.5 8.75] >>> find_idx_nearest_val(data_array, value=3) 2 >>> find_idx_nearest_val(data_array, value=3.5) 3 >>> find_idx_nearest_val(data_array, value=8.5) 7 >>> find_idx_nearest_val(data_array, value=10) 7 >>> find_idx_nearest_val(data_array, value=-1) 0
In case of a non-increasing array with equal candidates, the last one is returned
>>> data_array = np.array([3, 0, 2, 4, 2, 1]) >>> find_idx_nearest_val(data_array, value=2.1) 4 >>> data_array = np.array([3, 0, 2, 4, 2, 1, 2]) >>> find_idx_nearest_val(data_array, value=2.1) 6
If we change the first candidate, so it becomes the nearest, this one will be returned
>>> data_array = np.array([3, 0, 2.09, 4, 2, 1]) >>> find_idx_nearest_val(data_array, value=2.1) 2
Notes
In case that 2 or more items exist with the same distance from value, the last occurrence is returned
In case the value is outside the range of any value inside the array, either 0 or N-1 is return, with N the number of array elements of the input array
See also
get_nearest_indexThis function returns the first occurrence and only works for monotonically increasing arrays
- pymarine.utils.numerical.get_column_with_max_cumulative_value(data, regular_expression='.*')[source]
Find the column of a pandas DataFrame with the maximum cumulative value
- Parameters:
data (DataFrame) – Data frame with the columns
regular_expression (str) – Regular expression used to make a selection of columns to include. Default to ‘.*’, which means that all columns are included
- Returns:
The name of the column with the maximum cumulative value or None if no columns were found
- Return type:
str or None
Notes
Only the columns with a name obeying the regular expression are taken into account
An example of usage can be found in the fatigue monitoring software where we have data frames with damage over all the channels at a hot spots. If you want to obtained the channel with the maximum cumulative damage you can use this function
Examples
>>> import string >>> import pandas as pd >>> np.random.seed(0) >>> n_cols = 5 >>> n_rows = 10
Create a 10 x 5 data frame with random values with columns named as A, B, C, etc
>>> data_frame = pd.DataFrame(np.random.random_sample((n_rows, n_cols)), ... columns=list(string.ascii_uppercase)[:n_cols])
Obtain the name of the column with the maximum cumulative value
>>> get_column_with_max_cumulative_value(data_frame) 'D'
Obtain the name of the column with the maximum cumulative value only including colums A, B and C
>>> get_column_with_max_cumulative_value(data_frame, regular_expression="[ABC]") 'C'
- pymarine.utils.numerical.get_nearest_index(data, value)[source]
Find the index of the first occurrence of a value in a array with monotonically increasing values
- Parameters:
data (array_like) – Array with monotonically increasing numbers
value – Value of the point to give the index
- Returns:
Index of floor cell where the values in the array first exceed the value
- Return type:
Example
>>> data_array = np.linspace(0, 10, 8, endpoint=False) >>> print(data_array) [ 0. 1.25 2.5 3.75 5. 6.25 7.5 8.75] >>> get_nearest_index(data_array, value=3) 2 >>> get_nearest_index(data_array, value=3.5) 2 >>> get_nearest_index(data_array, value=8.5) 6 >>> get_nearest_index(data_array, value=10) 7 >>> get_nearest_index(data_array, value=-1) 0
In case of a non-increasing array an AssertionError is raised
>>> data_array = np.array([3, 0, 2, 4, 2.11, 1]) >>> try: ... get_nearest_index(data_array, value=2.1) ... except AssertionError as err: ... print("An assertion error was raised") An assertion error was raised
- Raises:
AssertionError – In case the array does not have monotonically increasing values
Notes
Only arrays with monotonically increasing values are allowed.
In case the nearest index of an arbitrary array is needed, find_inx_nearest_val should be used
In case the value a larger than any value in the data array, the maximum index is returned, while in case the value is smaller than any value in the data array, a zero is return
- pymarine.utils.numerical.get_parameter_list_key(parlist)[source]
Utility for the qtgraphs Parameter Item
- Parameters:
parlist – The parameter tree list (is an ordered list) contain all the values and current one
- Returns:
The name belonging to the current value
- Return type:
Notes
The parameter tree widget has a field ‘list’ in which a list of values is given with corresponding integers. The current integer belonging to the parlist is obtained by parlist.value() however, to get the associated value of the key field is less straightforward In this routine it is retrieved
- pymarine.utils.numerical.get_range_from_string(range_string)[source]
Analyse a range string to get the start, end and step and return a numpy array
- Parameters:
range_string (str, start:end:[step]) – String representing the start, end and step size of a range
- Returns:
1-D Array with points defined by range_string
- Return type:
ndarray
Examples
>>> get_range_from_string("0:10:2") array([ 0., 2., 4., 6., 8., 10.])
>>> get_range_from_string("0:7") array([ 0., 1., 2., 3., 4., 5., 6., 7.])
>>> get_range_from_string("3:4:0.2") array([ 3. , 3.2, 3.4, 3.6, 3.8, 4. ])
- pymarine.utils.numerical.loadmat(filename)[source]
Load a matlab data file with a complex data structure
- Parameters:
filename (str) – Name of the matlab file to import
- Returns:
Dictionary with the complex matlab data structure
- Return type:
Examples
To read a matlab file do
>>> import os >>> file_name = os.path.join("..", "data", "RAO_7.mat") >>> data = loadmat(filename=file_name)
Now have a look at the contents
>>> print_mat_nested(data) Key: DirRange Key: FreqRange Key: RAO Key: __globals__ Key: __header__ Key: __version__
The data stored in the Dir Range field can be accessed as
>>> data["DirRange"] array([ 0, 15, 30, 45, 60, 75, 90, 105, 120, 135, 150, 165, 180, 195, 210, 225, 240, 255, 270, 285, 300, 315, 330, 345], dtype=uint16)
Notes
This function should be called instead of direct spio.loadmat as it cures the problem of not properly recovering python dictionaries from mat files. It calls the function check keys to cure all entries which are still mat-objects
References
http://stackoverflow.com/questions/7008608/ scipy-io-loadmat-nested-structures-i-e-dictionaries
- pymarine.utils.numerical.make_2d_array_cyclic(data_2d, axis=0, add_constant=0.0)[source]
Makes a 2D array period by copying the first row toward the end
- Parameters:
data_2d (ndarray) – NxM array carrying the data. It is assumed that the data along one of the axis is periodic and that we want to make the array cyclic by copying the first row or column to the end
axis (int) – Axis to make periodic. Default = 0
add_constant (float) – Add this constant to the copy row or column to allow to add 2*pi or 360
- Returns:
Same 2d data array with one extra column (N+1 x M for axis == 0) or one extra row (N x M + 1 for axis == 1)
- Return type:
ndarray
Examples
First make some 2D data array using mesh grid containing the direction we want to make periodic
>>> directions = np.linspace(0, 360, 6, endpoint=False) >>> frequencies = np.linspace(0, 3, 3) >>> dd, ff = np.meshgrid(directions, frequencies) >>> dd array([[ 0., 60., 120., 180., 240., 300.], [ 0., 60., 120., 180., 240., 300.], [ 0., 60., 120., 180., 240., 300.]])
The array dd is periodic along the axis = 0 direction. So copy the first column to the end
>>> dd_periodic = make_2d_array_cyclic(dd) >>> dd_periodic array([[ 0., 60., 120., 180., 240., 300., 0.], [ 0., 60., 120., 180., 240., 300., 0.], [ 0., 60., 120., 180., 240., 300., 0.]])
We can do the same in case we have the transposed version, only we have to use the axis = 1 argument to pick the right axis
>>> dd_tr = dd.T >>> dd_tr array([[ 0., 0., 0.], [ 60., 60., 60.], [ 120., 120., 120.], [ 180., 180., 180.], [ 240., 240., 240.], [ 300., 300., 300.]]) >>> dd_tr_periodic = make_2d_array_cyclic(dd_tr, axis=1) >>> dd_tr_periodic array([[ 0., 0., 0.], [ 60., 60., 60.], [ 120., 120., 120.], [ 180., 180., 180.], [ 240., 240., 240.], [ 300., 300., 300.], [ 0., 0., 0.]])
For polar plotting it is required that the angle is increasing, which means that the last 0 needs to be 360. This can be established by using the add_constant option
>>> dd_periodic = make_2d_array_cyclic(dd, add_constant=360.) >>> dd_periodic array([[ 0., 60., 120., 180., 240., 300., 360.], [ 0., 60., 120., 180., 240., 300., 360.], [ 0., 60., 120., 180., 240., 300., 360.]])
Notes
This function can be used to ensure that a 2D data array with directions at one axis can be easily made periodic into the direction axis. This is a requirement to make a polar plot
- pymarine.utils.numerical.nans(shape, dtype=<class 'float'>)[source]
Create an array filled with
numpy.nanvalues- Parameters:
shape (tuple) – Shape of the array of nans to create
dtype (numpy.dtype, optional, default=float) – Type of the nan
- Returns:
Array filled with numpy.nan values
- Return type:
ndarray
Examples
>>> nans((2, 3)) array([[ nan, nan, nan], [ nan, nan, nan]])
- pymarine.utils.numerical.print_mat_nested(d, indent=0, nkeys=0)[source]
Pretty print nested structures from .mat files
References
http://stackoverflow.com/questions/3229419/ pretty-printing-nested-dictionaries-in-python
pymarine.utils.plotting module
Some functions to support with plotting in Python based on the matplotlib module
- pymarine.utils.plotting.analyse_annotations(annotation)[source]
Analyze the string annotation which compactly sets the properties of a label string such as position, size, and color.
- Parameters:
annotation (str) – label[@(xp, yp)[s10][a0][cRRGGBB]]
- Returns:
text (str), x position (float) y position (float), color (str), size (int), axis (int)
- Return type:
tuple of strings and integers
Notes
The annotation string can be just given as a label. This label optionally can be extended with a ‘@’ sign to include more information, like the location (xp,
yp), the size with s10, the color with c and the axis with a and a integer
The annotation format is mostly used as a compact way to provide information on the annotation via a user input file
When specifying a color with the c construct, make sure that you put the color at the end of the annotation string.
In the Parameter scription we used a hex formulation for the color like cFFAA00. However, also the color names like cblue or cred are allowed
In case you add chmc:red, the hmc definition of red will be used. Other hmc color definitions are:
blue
lightblue
red
lightred
green
darkcyan
Also we can add the xkcd color definitions as described in the matplotlib manual
Examples
Simple example of a label with all the properties set to default
>>> analyse_annotations("simple_label") ('simple_label', 0.0, 0.0, 'black', 12, 0)
place a label at position x=0.1, y=0.4
>>> analyse_annotations("more_complex@0.1,0.4") ('more_complex', 0.1, 0.4, 'black', 12, 0)
place a label at position x=0.8, y=0.9 with color red. Note that we need to add brackets around the location
>>> analyse_annotations("colored_label@(0.8,0.9)cred") ('colored_label', 0.8, 0.9, 'red', 12, 0)
place a label at position x=0.8, y=0.9 with color red. Note that we need to add brackets around the location
>>> analyse_annotations("small_label@(0.8,0.9)s8") ('small_label', 0.8, 0.9, 'black', 8, 0)
Place a label at position x=0.8, y=0.9 with color red and size 16. This time we need to add the color label at the end to extract it correcly
>>> analyse_annotations("large_colored_label_in_axis_2@(0.8,0.9)s16a2cred") ('large_colored_label_in_axis_2', 0.8, 0.9, 'red', 16, 2)
Finally lets show how you use more colours
>>> analyse_annotations("label@(0.8,0.9)s16a2chmc:red") ('label', 0.8, 0.9, '#DD2E1A', 16, 2)
As you can see, the hex code is returned.
To set the xkcd colors do
>>> analyse_annotations("label@(0.8,0.9)s16a2cxkcd:red") ('label', 0.8, 0.9, 'xkcd:red', 16, 2)
This color value ‘xkcd:red’ is understood by all matplotlib routines
- pymarine.utils.plotting.clean_up_artists(axis, artist_list)[source]
Remove all the artists stored in the artist_list belonging to the axis.
- Parameters:
axis (:class:matplotlib.axes.Axes) – Clean Artists (ie. items added to a matplotlib plot) belonging to this axis
artist_list (list) – List of artist to remove.
Notes
In case of animation of complex plots with contours and labels (such as a timer) we sometimes need to take care of removing all the Artists which are changing every time step. This function takes care of that. It also also ensured that we are not running out of memory when too many Artists are added
Examples
>>> from matplotlib.pyplot import subplots >>> from numpy.random import random_sample
Create a list which we use to store all the artists which need to be cleaned
>>> artist_list = list()
Create a plot of some random data
>>> fig, ax = subplots(ncols=1, nrows=1) >>> data = random_sample((20, 30)) >>> cs = ax.contourf(data)
Store the contour Artist in a list
>>> artist_list.append(cs)
Now clean it again
>>> clean_up_artists(ax, artist_list)
- pymarine.utils.plotting.clean_up_plot(artist_list)[source]
A small script to clean up all lines or other items of a matplot lib plot.
- Parameters:
artist_list – a list of items to clean up
Notes
Necessary if you want to loop over multiple plot and maintain the axes and only update the data. Basically this does the same as clean_up_artists
- pymarine.utils.plotting.clear_all_legends(axis)[source]
Remove all the legends from the current axis
- Parameters:
axis (list of :obj:AxesSubPlots)
- pymarine.utils.plotting.flip(items, ncol)[source]
Flip the legend list to get the lines in horizontal layout
- pymarine.utils.plotting.get_valid_legend_handles_labels(axis)[source]
Function to get all the labels from an axis which are not set to “None” (as a string!)
- pymarine.utils.plotting.set_limits(axis, v_min=None, v_max=None, direction=0)[source]
Set the x or y limits of the current axis and overwrite them if requested.
- Parameters:
- direction: int or str
Set limit on this axis. 0=x, 1=y, or pass “x” or “y”
- Returns:
[min, max] with the current x or y limits
- Return type:
Notes
If only the min or max is imposed, obtained the other limit from the current settings
- pymarine.utils.plotting.sub_plot_axis_to_2d(axis, n_rows=1, n_cols=1)[source]
Turn the axis return value of matplotlibs subfigures in a 2D list, regardless of the number of rows and columns
- Parameters:
- Returns:
type – new 2D list of the axis which can be referred to as axis[i_row][j_col]
- Return type:
Notes
The return axis of the matplotlib.subplots command is 2D in the case we have more than one row and more than one column. However, if there is only one row or col, the list will be 1D, for n_col = 1 and n_row = 1 the axis are directly returned. This scrips ensures that the list is always started as a 2D list such that the axis can be referred to with two indices: axis[i_row][j_col], also if we only have on row or columns
The practical usage of this function is to be able to write more generic code for which it is not known before hand how many rows and columns need to be generated (it may depend on the user input).
Examples
First create a matplotlib plot with only one row of two column
>>> import matplotlib.pyplot as plt >>> from numpy import (sin, cos, pi, linspace) >>> fig, axis = plt.subplots(nrows=1, ncols=2)
At this point, axis is a 1D list of 2 matplotlib.axis.Axes objects, one for each column. So we can plot like this
>>> x = linspace(0, 2 * pi, num=100) >>> l0 = axis[0].plot(x, sin(x)) >>> l1 = axis[1].plot(x, cos(x))
Now we want to turn that into a 2D list
>>> axis_2d = sub_plot_axis_to_2d(axis, n_cols=2)
Now, axis2d is a 2D list of Axes for 2 columns which are referred as axis[0][0] and axis[0][1], so we can plot
>>> l2d0 = axis_2d[0][0].plot(x, sin(x)) >>> l2d1 = axis_2d[0][1].plot(x, cos(x))
Note that we are referring to axis_2d as a 2D list with 2 indices in stead of one. This allows to create plots on the subplots returned axis in always the same way, which is handy if we don’t know beforehand how many rows and columns we have.